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ABSTRACT 

The final report describes the work performed by 
Riverside Research Institute (RRI) for the New York State Education 
Department in regard to the Reporting and Evaluation System for 
Occupational Education (RESOE). A brief introduction is followed by a 
section dealing with RESOE's design^ development^ and operation^ and 
including general discussions of improving management of occupational 
education^ and detailed discussions of RESOE^s two stages. State 1 
consists of the establishment of computer maintained enrollment^ 
f ollowup^ and other files which comprise the foundation of the RESOE 
information-processing capability. Stage 2 aims at developing RESOE 
into a useful instrument in support of management by evolving RESOE 
inro a versatile tool for program planning at State and local levels. 
The final section contains detailed discussions of activities 
performed to date: Stage 1 activities regarding dissemination and 
implementation^ and the RESOE brochure; and Stage 2 activities 
including statistical methods arsociated with followup mailing 
methods^ the details of the Instructional Support and Evaluation 
System (ISES) for occupational education^ and potential attendance 
subsystems and remote dara entry systems. An appendix contains a flow 
chart of the overall process^ and examples of forms^ envelopes^ and 
other materials used in RESOE implementation. (JR) 
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PREFACE 



During the past year, a Reporting and Evaluation System 
for Occupational Education (RESOE) has been further developed 
and implemented in a total of five of New York State's occupa- 
tional planning regions. RESOE implementation begins by 
establishing flexible^ computer-maintained student files for 
each participating education agency. These files are used to 
generate mandated reports for the State, and are available to 
meet other local, State, and Federal reporting and evaluation 
requirements in occupational education. 

The manner in which RESOE has been received by occupa- 
tional directors and their guidance personnel has been grati- 
fying. They have appreciated the underlying system concepts 
described in this report. They have appreciated that the 
system is here now, and ready to respond to many of their needs. 

Many occupational educators throughout New York State 
have actively cooperated with the State Education Department's 
Office of Occupational Education and Riverside Pesearch 
Institute (RRI) to develop and implement RESOE. Their commit- 
ment to RESOE *s implementation in their respective education 
agencies made orderly implementation possible. Their willing- 
ness to devote thought and effort to specifying their management 
information requirements, and to provide constructive criticism, 
has enabled RRI to improve RESOE so that the needs of education 
agencies can be served better. 

All who have provided this valued support cannot be cited 
here; however, RRI wishes to acknowledge the substantial con- 
tributions of the occupational administrators and guidance 
coordinators listed below who were designated to work directly 
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with RRI personnel implementing RESOE in their education 
agencies. 



Region 1 

Nassau BQCES 

I4r. James F. Wilde, Program Associate of Occupational 
Education 

Vocational Education and Extension Board (VEEB) 

Mr. Alfonso N. Fieri, Coordinator of Veterans' Training 

Region 2 

New York City 

Mr. George Quarles, Chief Administrator, Office of 
Career Education 

Mr. Norman Elliott, Director (Acting), Bureau of 
Business and Distributive Education 

Mr. Norman Watnick, Assistant Director, Bureau of 
Business and Distributive Education 

Region 8 

Broome, Delaware, Tioga BOCES 

Mr. James McElhatten, Administrative Assistant 

Delaware, Chenango, Madison, Otsego BOCES 

Mr. Robert Harold, Administrative Assistant for 
Program and Personnel 

Green, Delaware, Schoharie, Otsego BOCES 

Mr. Raymond J. Schoberlein, Director of Occupational 
Education 
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Region 9 

Tompkins / Seneca , Tioga BOCES 

Mr. Duane K. Ash, Guidance Coordinator 

Cayuga BOCES 

Mr. Charles Little, Director, Adult Education 

Syracuse City School District 

Mr. Hans Lang, Director, Occupational Education 
and Continuing Education 

Oswego BOCES 

Mr. Ronald Service, Director of Guidance 

Onondaga BOCES 

Mr. Charles West, Guidance Coordinator 

Cortland-Madison BOCES 

Mr. Francis Streeter, Director of Occupational 
Education 

Region 10 

Livings ton-Steuben-Wyoming BOCES 

Mr. George Couture, Director of Occupational Education 

Monroe BOCES #2 

Mr. Francis J. Day, Director of Occupational Education 

City School District of Rochester 

Mr. Edgar J* Hollwedel, Director of Occupational 
and Continuing Education 

Ontario-Seneca-Wayne-Yates BOCES 

Mr. Elliott Johnson, Director of Occupational Education 

Monroe County BOCES #1 

Mr. John McCormick, Assistant Superintendent 

Genesee-Wyoming BOCES 

Mr. Dale Post, Coordinator of Occupational Education 
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Region 11 

Schuyler-Chemung-Tioga BOCES 

Mr. George Dennis, Director of Occupational Education 

Steuben BOCES 

Mr. Rudolph Schneider, Director of Vocational Education 
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1. INTRODUCTION 

This final report describes work performed by Riverside 
Research Institute (RRI) for uhe New York State Education 
Department (SED) , Office of Occupational Education (OOE) under 
Contract C54490.. 

The report consists of three chapters: this brief intro- 
duction, a discussion of the general nature and background of 
RRI's work in support of occupational education, and a detailed 
description of work performed under the contract. In addition, 
an appendix is provided that contains a flow chart of the 
overall process, and examples of forms, envelopes and other 
materials used in RESOE implementation. 
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II. BACKGROUND AND GENERAL NATURE 
OF THE WORK 



xiRI's work for SED's Office of Occupational Education 
has the primary purpose of providing improved means of managing 
occupational education. 

A . Improving the Management of Occupational Education 

Management consists of three interrelated functions: 
planning, implementation and information processing. Work 
performed by RRI this year has involved design, development, 
and operation of the information-processing component for 
managing occupational education. This component is called the 
Reporting and Evaluation System for Occupational Education 
(RESOE) • 

Currently, RESOE is being developed to support two manage- 
ment processes associated with occupational education; Target- 
ing and Instructional Support and Evaluation • Both processes 



The Targeting and the Instructional Support and Evaluation 
System (ISES) are discussed in detail in various reports and 
other documents prepared by RRI for OOE. In particular, see 
the following reports: 

The Coordination of Program Planning and Evaluation Systems 
for Occupational Education, Volume II: A Targeting System for 
Occupational Education ; final report submitted to Dr» Robert S. 
Seckendorf , Assistant Commissioner for Occupational Education, 
SED, March 1972. 

The Coordination of Program Planning and Evaluation Systems 
for Occupational Education, Volume III; Monitoring System^or 
Occupational Education ; final report submitted to Dr. Robert S. 
Seckendorf, Assistant Commissioner for Occupational Education, 
SED, March 1972. 

In this report, reference is made to Targeting in Chapter 
III, Section B, Part 1, "Statistical Studies Associated with 
Follow-up," and to ISES in Chapt'-"*L- III, Section B, Tart 2, 
"ISES. " 
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are intended to tailor occupational progranis to improve the 
match between skills acquired by students and skills actually 
required for emp.i.oyment. Targeting matches programs offered 
in occupational education to the spectrum of available jobs; 
the Instructional Support and Evaluation System matches the 
content of specific programs to the needs of specific jobs. 
Operation of these- two processes is likely to be supported by 
the State, but local agencies are expected to operate, con- 
tribute to, participate in, and benefit from them. Further- 
more, because of the flexible computer processes used in RESOE, 
RESOE also will be capable of supporting a variety of other 
decision-making processes at state and local levels. 

An important design objective of RESOE is to provide an 
"automated," cost-effective means of generating mardated re- 
ports of various kinds, for example, reports of student enroll- 
ment for use in Part 2 of the "New York State Plan for the 
Administration of Occupational Education" submitted to the Board 
of Regents and the U. S. Office of Education.^ Much of RRI ' s 
work to date has involved developing such basic information- 
handling capabilities that represent the "foundation" of RESOE. 

Figure 1 schematically summarizes this brief discussion 
of the relationships between RESOE and the planning function. 

B. RESOE Development and Implementation 

The development and implementation of RESOE is proceeding 
in two stages. Stage I involves developing methods of proc- 
essing information necessary for fulfilling mandated reporting 



^ According to the terms of the Higher Education Act of 1965, 
Par? ? (Public Law 89-329) and the Vocational Education Amend- 



men 



ts of 1968 (Public Law 90-576). 
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FIG. 1 RESCE AND PLANNING 
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requirements on occupational education enrollments and comple- 
tions. Stage II involves adding capaibilities to RESOE to 
provide flexible, cost-effective data processing support for 
program planning at state and local levels. 

1. Stage I 

RESOE information rocessing applications are made 
possible by a computer-based, record-keeping system that con- 
tains files of several kinds. 

The first files to be developed are individual stu- 
dent files for storing "enrollment" information and "follow- 
up" information. "Enrollment" files contain information on 
currently enrolled students; "follow-up" files contain informa- 
tion on former students who have graduated from or have other- 
\\iise terminated occupational education. Concurrent with 
development of student files, local agency files are developed 
for storing information on local agency programs and classes. 
Stage I work^ to date has dealt primarily with establishing 
these files. File establishment involves implementing pro- 
cedures for obtainir9 information from students, and entering 
the information into computer storage. 

Once information for each student is entered into a 
file, any subset of data elements for each student can be with- 
drawn for aggregation with equivalent subsets from the files 
of other students. Thus, reports (presenting numbers of stu- 
dents in any category definable from the filed information) or 



The details of work performed are presented in Chapter III, 
Section A, of this report. 

During 1972, the previous contract year, these files were 
established for students and agencies in Region 9 and part of 
Region 1 (Nassau County) . In addition, a pilot effort was 

F/290-5-00 

12 



statistical analyses (involving any parameters on file) can 
be generated. Information does not need to be acquired for 
each display or analysis because, to the extent possible, 
required information is contained in existing discrete files. 
As new needs for data arise, data acquisition procedures and 
existing files can be expanded to meet these needs. 

Although RESOE ultimately will include the full set 
of files necessary for standard State and local program plan- 
ning, and will include optional sets specified by local users. 

RESOE file development has begun with establishing "enrollment" 

4 

and "follow-up" files for meeting mandated reporting require- 
ments associated with preparation, approval and submission of 
the "State Plan for the Administration of Occupational Educa- 
tion" to the U. S. Office of Education. Various procedures 
were used previously for obtaining and processing this informa- 
tion, but they have proven to be inadequate in a variety of 
ways. 

The previously used procedures had two serious 
deficiencies. Information was not obtained in the classroom 
and consequently data were inaccurate; information was provided 
in an "aggregated" form (numbers of students in various broad 

conducted in New York City to test the feasibility of estab- 
lishing equivalent enrollment files within the special con- 
straints existing in that large and complex school system. 

During 1973, this contract year, these basic files were 
established in Regions 8, 10, and 11, and they were maintained 
in Regions 1 and 9. The pilot study of the previous year in 
New York City led to full-scale establishment of student en- 
rollment files in 1973. 

^ According to the Higher Education Act of 1965, Part F (Public 
Law 89-329) and the Vocational Education Amendments of 1968 
(Public Law 90-576) . 
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categories) and consequently, the data often were not useful 
for purposes other than the specific report for which they 
were obtained. RESOE was designed to correct these inadequa- 
cies. Individual student data are obtained, and files maintain 
data in a "disaggregated" form. Thus, standard reports can be 
prepared readily, but a capability exists to preparo non- 
standard or unanticipated reports using part or all of the same 
data. 

RESOE Stage I "enrollment" and "follow-up" files 
contain information for each student. Enrollment files contain 
information on each student currently enrolled in each occupa- 
tional program in each local agency. Similarly, follow-up 
files contain information on each formerly enrolled student. 

Enrollment files contain student name, sex, ethnicity, 
level (adult or secondary) , type (regular, disadvantaged, or 
handicapped) , program name (and code) , program type (regular 
or cooperative), and, for adult students, the students* purpose 
in taking occupational education. Each student is identified 
by an identification nximber, and his class and local agency 
are specified. This information is used to prepare state-wide 
and local enrollment reports. In addition, some categories 
of information, such as student address, currently are included 
because of widespread interest in them by local administrators. 

Information in follow-up files includes student name 
and current educational or occupational activity. This infor- 
mation is combined with information in each student's enroll- 
ment file to prepare state-wide and local follow-up reports. 

Figure 2 summarizes these basic Stage I processes. 
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A central facility (RRI at present) prepares and 
distributes materials used in the enrollment and follow-up 
surveys (forms or questionnaires, ins ructions, envelopes). 
Students or former students enter information on forms or 
questionnaires. Forms or questionnaires are returned to the 
central facility where information is entered into computer 
files . 

Using the filed information, the computer prepares 
state-wide siimmary reports for SED, agency-wide summary reports 
for local education agencies (LEAs, including BOCES and school 
districts) and reports for each LEA which list' individual stu- 
dents and pertinent information for each occupational education 
student at the local agency. (The general process depicted 
in summary form here is explained in detail for enrollment and 
follow-up later in Chapter III, Section A, of this report.) 

2. Stage II 

As stated earlier in this report. Stage I represents 
the foundation of the RESOE information processing capability. 
Stage II represents additional capabilities that make RESOE a 
complete, functioning subsystem. Stage II is intended to 
develop RESOE into a useful instrument in support of management 
by evolving RESOE into a versatile tool for program planning 
at State and local levels. 

5 

In RRI's original proposal for work to be done this 
contract year, some effort was to be devoted to developing 



The Coordination of Program Planning and Evaluation Systems 
for Occupational Education , proposal submitted to Dr. Robert S* 
Seckendorf, Assistant Commissioner for Occupational Education, 
January, 1973. 
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stage II activities. The original work plan was modified in 
response to an OOE request to accelerate effort to refine Stage 
I this year, and to prepare for implementing Stage I processes 
state-wide by 1974. (That is, Stage I processes are to obtain 
and process enrollment information throughout the State in the 
school year 1974-75.) To accommodate this shifted emphasis 
in work, OOE suggested that some of the effort devoted to 
Stage II work be reduced during the current contract year. 

7 

In particular, Monitoring and Targeting efforts 
were reduced. The basic concept and design for Targeting had 

g 

been developed by RRI and described in a 1972 RRI report to 

SED. The basic design for Monitoring in occupational educa- 

9 

tion also iiad been developed and described in a 1971 report 
to SED. Although this work was not developed as thoroughly as 
the targeting concept and design, RRI has acquired considerable 
experience in the information processing aspects of Monitoring 



See the progress report submitted to Mrs. Florence Sutler 
dated May 14, 1973. 

7 

"Monitoring" was the term first used in the early phases of 
RRI's work for OEE and has been replaced by the term Instruc- 
tional Support and Evaluation Subsystem (ISES) , which currently 
is a more accurate, descri tive term (See page 2 of this report) 

g 

The Coordination of Program Planning and Evaluation Systems 
for Occupational Education, Volume II: A Targeting System for 
Occupational Education ; final report submitted to Dr . Robert 
S~. Seckendorf , Assistant Commissioner for Occupational Educa- 
tion, SED, March 1972. 

9 

The Coordination of Program Planning and Evaluation Systems 
for Occupational Education, Volume III: Monitoring System for 
Occupational Education ; final report submitted to Dr. Robert 
S. Seckendorf, Assistant Commissioner for Occupational Educa- 
tion, SED, March 1972. 
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in a project performed for the Guilderland (New York) Central 

School District. The Guilderland project involved designing 

and implementing an information -handling system to support 

instruction of mathematics and reading. This system is called 

an Ins tr actional Support System (ISS) and is described in 

several RRI reports and proposals. "'"^ ' During this year the 

concept of ISS in occupational education was described in a 
12 

report prepared by RBI for SED. 

Subsequently, based on experience explaining RESOE 

to local educators throughout the State, RRI and OOE perceived 

13 

the need to develop a brochure to provide information on 
RESOE to local educators who are to implement RESOE in their 
agencies during the 1974-75 school year. Accordingly, to 
accommodate the work required to prepare the brochure in time 
for it to be used in disseminating RESOE during the early 
part of 1974, the inventive aspects of inferential statistical 
efforts and the detailed trade-off analysis associated with 



The Coordination of Program Planning and Evaluation Systems 
for Occupational Education, Volume III: Monitoring System for 
Occupational Education ; final report submitted to Dr* Robert S. 
Seckendorf , Assistant Commissioner for Occupational Education, 
SED, March 1972. 

-^-^ The Development of an Instructional Support System . Renewal 
proposal, submitted to the Guilderland Central School District 
Number 2, Guilderland, New York, 1972. 
12 

Improving the Management of Instruction in Occupational 
Education, Concept Paper submitted to Dr. Robert S. Seckendorf, 
Assistant Commissioner for Occupational Education, SED, 
September, 1973 

-^^The rationale underlying this perception and the concept and 
evolution of the RESOE brochure are discussed later, in 
Chapter III, Section A, of this report. 
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the attendance "package" were postponed by agreement with the 

14 

contract monitor. 

As a consequence of these two revisions in RRI's 

work plan, Stage II efforts this year have focused on the 

15 

following tasks : 

• Formulating design concepts associated with 
monitoring, termed Instructional Support and 
Evaluation System (ISES) ; reviewing and evalua- 
ting existing efforts within SED related to 
ISES for occupational education; 

© Performing statistical studies related to 
follow-up that are limited to interpreting 
the results of follow-up mailing method 
experiments; 

• Describing the existing requirements for 
attendance data and methods of obtaining and 
processing them; formulating preliminary 
functional specifications of an attendance 
"package" for use in occupational education; 

• Undertaking a demonstration of data entry using 
remote terminals to examine the human performance 
aspects of remote data entry, in particular, the 
feasibility of reducing the error rate in entered 
data (and consequently, editing costs) by trans- 
ferring editing and error correction functions 

to local agency personnel* 



14 



See the letter to Mrs. Florence Sutler, dated November 8, 1973 



■'■^The work performed is discussed in detail in Chapter III, 
Section B, of this report. 
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III. WORK PERFORMED THIS CONTRACT YEAR 

This chapter describes the work performed this year in 
two sections. Section A is devoted to a detailed discussion 
of work related to Stage I; Section B describes work related 
to Stage II in terms of concepts, design specifications, and 
analyses . 

A. Stage I 

This section of the Final Report describes RESOE Stage I 
dissemination activities, data processing operations, bro- 
chure preparation, and remote terminal demonstration during 
the past contract year. 

1. RESOE Stage I Dissemination Activities 

During the past contract year, introductory meetings 
were held at Regions^^ 8, 10, and 11 to explain RESOE pur- 
poses and procedures to BOCES and school district superinten- 
dents. Presentations at these meetings were given by Dr. 
Robert Seckendorf and Mrs. Florence Sutler of OOE with support 
from RRI. At each introductory meeting, superintendents 
expressed interest in and provisional support of RESOE, and 
each superintendent designated a "coordinator" (a person to 
assume responsibility for implementing RESOE in agencies 
under the superintendent) . 

Following the introductory meetings, implementation 
meetings were held at the BOCES in Regions 8, 10, and 11, and 



The names and locations of agencies in regions in which 
RESOE has been implemented are provided in the Preface, pages 
iv to vi. 
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the City of Rochester. These meetings were held for the 
purposes of introducing designated coordinators in local 
agencies to existing RESOE Stage I capabilities and to 
anticipated RESOE (Stage I and Stage II) capabilities. At 
these meetings, coordinators were asked to implement basic 
RESOE processes (establishing individual student files) in 
their respective regions. Their responsibility*'-^ as coordina- 
tors for this first year of implementation in Regions 8, 10, 
and 11 includes the following primary functions: 

• Designating a counterpart to the RESOE coordina- 
tor at the district or school level; 

• Acting as RESOE liaison with RRI/SED; 

• Acting as liaison with affiliated local 

in distributing and collecting RESOE enrollment 
forms . 

These regional implementation meetings typically 
were attended by occupational education directors, guidance 
counselors, and representatives from RRI and SED. Informal 
openended discussions were invited after a more formal RESOE 
presentation was made. The meetings were successful, par- 
ticularly in two important respects: regional educational 
managers learned about RESOE capabilities, and what they 
should do to implement basic RESOE processes; they were able 
to obtain answers to their questions, make suggestions and 
otherwise influence the implementation of RESOE in their 
respective regions. 

During the current contract year, plans for RESOE 
Stage I operations were formulated to include the remaining 



See Appendix A for a more detailed description of what this 
responsibility entails. 
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regions in the State (Regions 3, 4, 5, 6, 7, 12, 13, and part 
of Region 1). Two types of meetings were planned: introduc- 
tory, "kick-off" regional meetings to introduce BOCES and 
city superintendents to RESOE, and shortly thereafter, re- 
gional implementation meetings with designated BOCES or city 
RESOE coordinators to operationalize RESOE procedures in 
their localities. 

2 . RESOE Stage I Implementation Activities 

Implementation during this past contract year con- 
cerned basic Stage I RESOE enrollment and follow-up procedures. 

a. RESOE Enrollment 

The overall enrollment procedure is described 
below in terms of its two constituent phases: data acquisi- 
tion and data processing. 

1. ) Data Acquisition 

Enrollment materials were produced and 
distributed in Regions 1, 2, 8, 9, 10, and 11. These mate- 
rials included enrollment forms, instructions, envelopes and 
covering letters. 

Secondary and adult occupational students 
were included in RESOE enrollment. 

Slightly different enrollment procedures 
were implemented for Region 1 (Nassau County) , Region 2 (New 
York City) , and Regions 8 through 11 ("upstate" areas including 
the cities of Rochester and Syracuse) . 



•'■^See Appendix B for samples of RESOE enrollment materials 
used during the past contract year. 
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a, ) Procedures in Regions 8/ 9, 10, and 11 



Figure 3 depicts the enrollment data 
acquisition procedures used in Regions 8, 9, 10, and 1] Ex- 
cept for details, this procedure was equivalent to that used 
in other regions. 

Labelled envelopes containing enroll- 
ment materials were sent to specific classes in each school or 
BOCES in Regions 8 through 11. Information for labelling was 
obtained in the following manner. 

During the spring and summer of 1973/ 
BOCES and the cities of Rochester and Syracuse provided RRI 
with pre-enrollment information that listed occupational educa- 
tion classes for each BOCES or local school. Each course to 
be given was identified by program title, level (adult or 
secondary) , program type (regular or cooperative) / and the 
teacher assigned to each class. 

This pre-enrollment information v-zas 
entered into computer files and subsequently was printed by 
the computer on the label of each envelope. In this way each 
envelope could be sent to a specific class, and all class- 
associated information required for subsequent data processing 
was presented on the envelope label. In addition, to accom- 
modate new classes, several extra envelopes having only local 
agency name and location code printed on the label were sent 
to each school and BOCES. In the case of new classes, a school 
or BOCES official entered missing information on these partially 
labelled envelopes prior to forwarding them to classrooms. 

In the classroom, teachers read the 
instructions, distributed enrollment forms to students, super- 
vised the completion of forms, collected forms and returned 
them in the envelope to the designated school or BOCES official. 
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\ COLLECT PRE-ENROLLMENT 
DATA FROM REGIONAL, 
CITY AND LOCAL AGENCIES 
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FILES 



COLLECT STUDENT ENROLLMENT 
DATA IN THE FALL 
BASED ON PRE-ENROLLMENT FILES, 



t 

ESTABLISH COMPUTER-BASED 
STUDENT ENROLLMENT FIL ES 

T 

SUMMARIZE ENROLLMENT SURVEY 
DATA TO MEET MANDATED 
REPORTING REQUIREMENTS 



SEND REPORTS TO LOCAL 
EDUCATION AGENCIES FOR 
THEIR APPROVAL 



SEND ANNUAL MANDATED REPORTS 
TO THE STATE EDUCATION 
DEPARTMENT AND TO LOCAL EDUCATION 
AGENCIES 



FIG. 3 SEQUENCE OF STEPS IN RESOE'S BASIC 
STUDENT ENROLLMENT PROCESS 
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In Rochester and Syracuse, a city 
occupational education director distributed materials to local 
schools, and, upon completion of the forms, returned the 
materials to RRI, In the case of BOCES-served areas^ the BOCES 
occupational education coordinator performed these tasks • 

) Procedures in Region 1 

BOCES in Region 1 (Nassau County) 
utilized RESOE pre-enrollment and enrollment procedures. How- 
ever, because the BOCES and school districts have not yet 
developed an effective manner of implementing RESOE processes 
in local schools, local schools in this region continued to 
use OEP-1 forms""-^ for enrollment this year. RESOE enrollment 
procedures used by the BOCES in Region 1 were the same as those 
in Regions 8 through 11. 

c. ) Procedures in Region 2 

In Region 2 (New York City) , the over- 
all approach was similar to that used in other regions. How- 
ever, minor differences existed because of local requirements. 
For example, smaller and simpler forms and envelopes were 
used in New York City than were used in Regions 1 and 8 through 
11. Enrollment envelopes did not contain preprinted class- 
or program-related infomiation; this information was provided 
by an administrator in each school. Student names did not 
appear on enrollment forms submitted to RRI. 



■^^OEP-1 forms are used to obtain secondary and adult level 
enrollment rlata in the Occupational Education Data System (OEDS) 
that RESOE is to supplant. 
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New York City provided RRI with pre- 
enrollment information necessary to distribute enrollment 
materials to schools and other educational <igencies in Region 
2. Distribution information consisted of names of schools and 
other agencies within the city that were to receive enrollment 
materials, and the quantities of materials to be receive.d by 
each. 

2 . ) Data Processing 

Data obtained through RESOE enrollment 
procedures are being used to produce two kinds of reports. 

One kind of report, referred to as "OEDS- 
style," will present aggregated individual student data in a 
format equivalent to the format used in OEP-1 forms. Aggre- 
gated data in this format then will be merged with data ob- 
tained using OEP-1 forms (used in regions where RESOE is not 
yet implemented) and incorporated into the state-wide OEDS 
secondary- and adult-level enrollment report. The Office of 
Occupational Education at SED will receive the state-wide 
enrollment data aggregated from RESOE and OEP-1 forms. In 
aduition, each local agency also will receive a report of this 
kind. However, "OEDS-style" reports sent to each local agency 
will be based on data pertinent only to students enrolled at 
the local agency. 

The second kind of report is designed to 
provide local agencies with individual student enrollment 
information for program management, guidance, and administrative 
purposes. This kind lists individual students by each occupa- 
tic lal classroom and by agency; it presents information per- 
taining to each student (such as address, telephone number, 
grade level, employment status, etc.); it summarizes individual 
student data in tables for each class and agency student listing. 
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A student will be listed for each class in which he is enrolled; 
thus in these reports, individual students may be counted more 
than once. 

In summary, the Office of Occupational 
Education at SED will receive enrollment data obtained using 
RESOE enrollment materials and OEP-1 forms in an enrollment 
report that aggregates data for all occupational students in 
the State. Each local agency will receive "OEDS-style" reports 
that aggregate data for students in that agency, and reports 
that present individual student data and aggregate these data 
in summary tables. 

The general data processing procedures 
discussed above were used in all regions. Hov/ever, in New York 
City, a minor additional task has been undertaken Dy RRI at 
the request of Mr. Norman Elliott, the RESOE coordinator in 
New York City. Mr. Elliott requested enrollment data from 
some New York City educational agencies offering courses related 
to occupational education, but that were not "occupational" 
in the sense of the SED definition. These data are not to be 
included in the state-wide report but are being processed 
separately, at RRI's expense, for use by occupational educa- 
tion administrators in New York City. 

b. RESOE Follow-Up 

RESOE follow-up was performed in local agencies 
in which enrollment processes and files had been established 
during the preceding year; these agencies are in Regions 1 and 
9. The overall follow-up procedure is described below in 
terms of its two constituent phases: data acquisition and 
data processing. 
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1. ) Data Acquisition 

RESOE follow-up data were obtained in 
Region 1 (Nassau County) and Region 9 (Syracuse and environs) . 
However, differences existed in the manner of obtaining data. 

a. ) Procedures in Region 9 

Figure 4 depicts follow-up procedures 

used in Region 9. 

In Region 9, follovz-up questionnaires 
were mailed to foinner students (graduating and non-completing 
students) . Follow-up data were obtained on these question- 
naires from individual students. 

Information necessary for mailing the 
questionnaires and for relating follow-up data to data already 
in enrollment files was obtained during the spring of 1973. 
Occupational education students were asked to enter informa- 
tion pertaining to their expected address and activities on 
a form entitled "Student Address Form.*' This information was 
compiled m a printout that was sent to school and BOCES 
officials in the region for approval, verification, and error 
correction. 

Questionnaires were mailed to former 
students using three mailing modes: first-class, hand-stamped; 
first-class, machine-stamped; and third-class (bulk rate) . 
These three mailing modes were used to compare questionnaire 
return rates as a function of mailing mode. Third-class and 
first-class, machine-stamped modes were used randomly in about 



^ See Appendix B for samples of RESOE follow-up materials used 
during the past contract year. 



F/290-5-00 



- 21 



^COLLECT PRE-FOLLOW-U^ 
DATA FROM 
LOCAL AGENCIES 



ESTABLISH COMPUTER-BASED 
STUDENT FOLLOW-UP LISTS 
BASED ON DATA FROM 
STUDENT ENROLLMENT FILES 



COLLECT FOLLOW-UP/ 
.SURVEY DATA FROM 
-FORMER STUDENTS^ 
\ IN FALL 



SUMMARIZE DATA FROM 
STUDENT FOLLOW-UP SURVEYS 
TO MEET MANDATED 
REPORTING REQUIREMENTS 



SEND TO LOCAL EDUCATION 
AGENCIES LIST OF STUDENTS 
NOT RESPONDING IN STANDARD 
SURVEY 




LOCAL EDUCATION AGENCIES 
ADMINISTER OPTIONAL-'' LOCAL 

FOLLOW-UP FOR STUDENTS 
S,NOT RESPONDING IN STANDARD^ 
SURVEY 



7 



L 



SEND MANDATED FOLLOW-UP 
REPORTS TO STATE EDUCATION 
DEPARTMENT AND TO LOCAL 
EDUCATION AGENCIE 



SUMMARIZE DATA FROM 
OPTIONAL* LOCAL 
FOLLOW-UP SURVEY 



I 



SEND OPTIONAL* LOCAL 

F0LL0V7-UP REPORTS 
TO LOCAL EDUCATION 
AGENCIES 



PIG 4 SEQUENCE OF STEPS IN RESOE'S BASIC STUDENT FOLLOW-UP 
PROCESS (^OPTIONAL ACTIVITIES UNDERTAKEN AT LEA COST) 
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one- third of the mailings; equal numbers of mailings were made 
using these two modes. Based on this experiment, future sur- 
veys that may be required can be conducted using the most "cost- 
effective" mode. 

In an attempt to insure an adequate 
return of questionnaires, former students not returning their 
completed questionnaires by a certain date v/ere sent a second 
questionnaire. The effectiveness of the second mailing in 
increasing the overall return rate will be measured. Based on 
these measurements, future surveys can be conducted using the 
more "cost-effective" method, i.e., the single or double 
inailing • 

b . ) Procedures in Region 1 

In Region 1 (Nassau County) , follow- 
up data were obtained via a questionnaire prepared and dis- 
tributed by the Nassau BOCES . This questionnaire utilized 
the format and specific questions of the RESOE questionnaire. 
As a consequence, data processing procedures for RESOE and 
Nassau BOCES forms is identical. However, because the Nassau 
BOCES (not RRI) mailed questionnaires to former students in 
Region 1, no mailing experiments were performed in this region* 

2 . ) Data Processing 

Two general kinds of reports (analogous 
to those prepared from enrollment data) are being prepared 
based on data obtained from follow-up questionnaires. The first 

kind will present aggregated data in a format equivalent to that 

2 1 

used in OEP-2 forms using "unduplicated" counts of former stu- 



OEP-2 forms are used to obtain secondary and adult level 
completions data in the Occupational Education Data System 
(OEDS) that RESOE is to supplant. 
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dents. These aggregated data then will be incorporated, with 
data obtained on OEP-2 forms (used in regions where RESOE has 
not been implemented) , into the state-wide follow-up report. 
The second kind will be prepared for local agencies and will 
present individual student data in lists organized by classroom 
and agency, and will aggregate the data in summary tables for 
each list. The state-wide aggregation will be submitted to the 
Office of Occupational Education at SED; each local agency will 
receive two kinds of reports on students enrolled in the agency 
reports presenting aggregated data in the style of OEP-2 forms, 
and reports presenting data on individual students with summary 
data tables at the end of each list. 

Data obtained in the spring and those ob- 
tained through the follow-up survey are being examined to 
compare student employment and education expectations of last 
spring with their actual activities of this fall. These com- 
parisons can provide a basis for estimating the validity of 
expectation or "placement" data obtained prior to graduation. 
If placement data appear to correlate positively with follow- 
up data, placement or expectation data may provide a means of 
increasing the confidence of estimates regarding the activities 
of former students in cases where follow-up questionnaire 
returns are low or sampling techniques are used in the follow- 
up survey. 

In addition to reports based on returned 
ques ionnaires, local agencies will be sent a computer print- 
out that lists former students (with their addresses and 
telephone numbers) from whom questionnaires have not been 
received. Local agencies also will be sent a set of follow-up 
questionnaires that are blank except for student mailing labels, 
Using these questionnaires, local agency personnel, at their 
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option, may extend the follow-up survey. For example, they 
might use the telephone numbers given on the printout to 
interview former students using the telephone while entering 
responses on the appropriate blank questionnaire. Alternatively, 
they might choose to mail the questionnaires to studente. 

Data pertaining to the mailing mode and 
mailing number experiments described above will be presented 
to the Office of Occupational Education upon analysis of the 
data. 

c. OEP Form Distribution and Data Processing 

RRI performed data processing of enrollment 
(OEP-1 and OEP-4^^) and follow-up (OEP-2 and OEP-5^^) data for 
all secondary and adult and post-secondary students who were 
not included in RESOE enrollment and follow-up surveys. In 
addition, in October, RRI distributed OEP-2 and 3 forms to 
educational agencies in regions currently using RESOE for 
enrollment procedures (OEP-1 forms to LEAs in Region 1) , and 
OEP-3 forms to regions currently using RESOE enrollment and 
follow-up procedures. 

Figure 5 depicts procedures used to distribute 
OEP forms and process OEP data. 

RRI distributed OEP-1, OEP-2, and OEP-3 materials 
directly to local agencies in regions in which RESOE activities 
currently are being pursued, i.e., in Regions 1, 2, 8, 9, 10, 



OEP-4 and OEP-5 forms respectively, are post-secondary level 
equivalents to OEP-1 and OEP-2 forms described earlier. OEP-3 
forms obtain personnel data for personnel not included in the 
Basic Education Data System (BEDS) at the secondary and adult 
levels, and OEP-6 forms are the post-secondary level equivalent 
to OEP-3 forms. 
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and 11. Specifically, Region 1 received OEP-1 and 0EP'-2 for 
use in local schools (but RESOE materials were used in BOCES) 
and OEP-3 forms for use in local schools and BOCES; Regions 2, 
8/ 10, and 11 received OEP-2 and OEP-3 forms; Region 9 received 
only OEP-3 forms. (SED distributed OEP-4, OEP-5, and OEP-6 
forms to all post-secondary facilities in these regions.) 
The purpose of RRI's distributing OEP forms in these regions 
was to prevent confusion that might arise if local agencies 
v;ere to receive a variety of materials intended to serve 
ostensibly identical purposes. RRI sent only those specific 
materials that each local agency required to fulfill its 
reporting, requirements. Redundancy was eliminated, and the 
resulting potential for confusion was reduced. 

RRI is processing data obtained on OEP-1, OEP- 
2, OEP-4, and OEP-5 forms throughout the State. SED is sending 
to RRI OEP-1 and OEP- 2 forms used in regions other than 1, 2, 
8, 9, 10, and 11, and all OEP-4 and OEP-5 forms. RESOE data 
will be incorporated by RRI into state-wide enrollment and 
follow-up reports. 

3. RESOE Brochure 

In light of the shift of emphasis to Stage I imple- 
mentation, the "Layman's Guide," a document previously prepared 
to aid in disseminating RESOE concepts and operations, did not 
appear to be sufficiently simple or direct for effectively 
describing RESOE Stage I processes to a readership primarily 
consisting of educators in local agencies. In addition, the 
"Layman's Guide" was intended to be a user's guide regarding 
ways to utilize the full spectrum of flexible data processing 
capabilities in RESOE based upon SEED; using the basic Stage I 
processes does not require such knowledge. Consequently, 
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RRI's work plan was revised upon agreement with the contract 

monitor to develop a new document focused on Stage I for use 

in dissemination. Accordingly, preparation of this document, 

referred to as the "RESOE BjL'ochure," has begun and will be 

23 

completed presently. 

The RESOE Brochure is intended to be an instrument 
for aiding dissemination of RESOE on the BOCES and school dis- 
trict level. It is directed toward a readership consisting 
mostly of local educators at the secondary and adult levels of 
occupational education (administrators primarily, but also 
teachers) • in addition, it is likely to be read by parents 
and students, by educators at the State level or in post- 
secondary institutions, by members of the executive or legis- 
lative branches of the State government, and by members of 
federal agencies or state agencies in other states • Focused 
on the primary readership, but with this broad potential reade 
ship in mind, the Brochure is being designed to attract and 
hold the attention of the entire spectrum of readers, and to 
inform them regarding the significance, general concept, and 
mode of operation of RESOE with emphasis on Stage I processes • 

B . Stage II 

Work has been performed this year on developing RESOE 
capabilities associated with functions intended to support 
program management in occupational education or to upgrade 
basic Stage I processes. Effort has been directed toward the 
following activities: developing design concepts for an 
Instructional Support and Evaluation System for Occupational 



As of the date of printing this final report, "mechanicals" 
of the Brochure have been submitted to OOE for printing. 
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Education (ISES) , developing statistical methods for analyzing 
and extending the usefulness of follow-up data, developing 
design concepts for an attendance system, and undertaking a 
demonstration and feasibility test of data entry using a 
remote terminal . 

1. Statistical Methods Associated with Follow-up 
Mailing Methods 

Follow-up questionnaires were mailed to former, 
secondary and adult, occupational education students using 
three mailing modes in an initial mailing. Former students 
who did not reply to the initial mailing were sent a second 
questionnaire. In addition, some former students, not included 
in the standard, follow-up, mailing file established in the 
spring, were sent questionnaires at the request of educational 
administrators. This section ' addresses the return rates 
associated with these mailings. 

a . Return Rate as a Function of Mailing Mode 

In the first mailing, a total of 6,732 question- 
naires were sent to former students and 1,733 were completed 
by the recipients and returned to RRI. Thus the overall return 
rate was 26%. 

The questionnaires were sent using three methods 
of mailing: first-class, machine-stamped; first-class, hand- 
stamped; and third-class or bulk rate. In the first-class, 



^This section discusses mailing and return aspects of the 
follow-up survey only; it does not address the student data 
derived from the survey. These data and analyses of them will 
be present in separate comprehensive reports as soon as the 
analyses are completed. 
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machine-stamped mode, 3,945 questionnaires were sent, and 992 
were completed and returned; thus the return rate was 25%. In 
the first-class, hand-stamped mode, 1,401 questionnaires were 
sent, and 387 were completed and returned; thus the return rate 
was 28%. In the bulk-rate mode, 1,486 questionnaires were 
sent, and 354 were completed and returned; thus the return rate 
was 25%. 

Although the differences in return rate may not 
be significant, the first-class, hand-stamped mode has the 
highest return rate. 

b. Return Rate Increase as a Function of the Second 
Mailing " 

The mailing of questionnaires to former students 
who did hot reply in the first mailing increased the overall 
return rate significantly* 

In the second mailing, 4,999 questionnaires were 
sent (by the first-class, machine-stamped mode for convenience), 
and 864 were returned; thus the return rate for the second 
mailing alone was 17%. However, the second mailing increased 
the overall return rate (as number of former students replying 
per number of former students sent questionnaires) to 39%. 

c. Effect of Undeliverable Questionnaires 

Several questionnaires were returned to RRI by 
the postal service marked "undeliverable." Although the total 
undeliverable questionnaires was 788, data currently are not 
available to identify the number of individuals to whom question- 
naires were not deliverable. Some former students may have been 
sent more than one questionnaire either because they were sent 
questionnaires in two mailings or because they were sent more 
than one questionnaire as a result of having been enrolled 
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in more than one program. (However, the latter consideration 
also implies that fewer than 6,732 individuals were sent 
questionnaires. Because some individuals received more than 
one questionnaire, return rates are actually higher than those 
calculated based on the numbers of questionnaires sent and 
returned. ) 



made that questionnaires were undeliverable to 788 former stu- 
dents (i.e., that 6,732 minus 788 individuals were sent question- 
naires) , the overall return rate (taking both mailings into 
account) was 44%. 



fonner students in response to special requests of educational 
administrators (in contrast to standard RESOE procedures) are 
not included in the discussion above. The total number of 
questionnaires sent in this category is 588. Of these, 31 were 
completed and returned; thus the return rate in this category 
is only 5%/ significantly less than the return rate for question- 
naires sent through standard procedures. 

e . Tentative Conclusions 

The following tentative conclusions can be drawn 
from the data given above: 



In the extreme case in which the assumption is 



Special Follow-up Mailings 



Questionnaires sent, completed and returned by 



1.) 



The difference between mailing modes was 
slight, therefore, because the cost of bulk 
mail is far less than that of first-class 



mail, bulk mail would be the most cost- 
effective mailing mode in future mailings* 



2.) 



The second mailing increased the return rate 



significantly (ignoring the effect of "un- 
deliverables , " it increases the rate from 
26% for the first mailing alone to 39% 
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in combined mailings); however, the cost 
for two mailings is very nearly double 
the cost of a single mailing. Because 
doubling the cost does not double the 
return rate, the second mailing is not 
cost-effective in terms of return rate 
alone; on the other hand, simple return 
rate may net be an appropriate or compre- 
hensive criterion for effectiveness. 



f . Data Summar y 

The table below depicts the results described 

above . 



Mailing 


Number 
Sent 


Number Completed 
and Returned 


Percent 
Returned 


Firs t (machine- 
stamped) 


3,945 


992 


25 


First (hand- 
stair.ped) 


1,'101 


387 


28 


First(bulk) 


1,486 


354 


25 


First(total) 


6,732 


1, 733 


26 


Second (total) 


4,999 


864 


17 


Overall (gross , 
i.e., including 
"undeliverables " ) 


6,732 


2,597 


39 


Overall (net, 
i.e., discounting 
"undeliverables") 


5,944 


2,597 


44 


Special 


588 


31 


5 
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2. ISES for Occupational Education 

An Instructional Support and Evaluation System (ISES) 
for occupational education can provide a means for more effec- 
tively using instructional resources. ISES can do this by 
providing support for managing (i.e., designing, implementing, 
evaluating, and redesigning) occupational programs in terms 
of their objectives and content, and by provid.rng support for 
managing (i.e., planning, monitoring, controlling, and re- 
planning) delivery of instruction in the classroom. 

ISES efforts, as discussed in this report, include 
the following two major components: 

• Program analysis and design; 

• Information processing. 

Program analysis and design strives to identify discrete occu- 
pational education programs and their component subprograms, 
to formulate program objectives in terms of terminal objectives 
and their component intermediate objectives, and to design 
modular curricula in terms of their module objectives, instruc- 
tional content, activities, criterion-referenced tests and 
resources. Information processing is required to put the 



The discussion of this section draws upon experience g^.ined 
by RRI in work performed for the Guilderland Central School 
District since 1971 in developing an instructional support 
system for reading and mathematics. (See for example The 
Development of an Instructional Support System , renewal pro- 
posal, submitted to the Guilderland Central School District 
Number 2, Guilderland, New York, 1972.) It also draws upon 
a concept paper prepared for Dr. Robert Seckendorf in September 
of 1973 entitled. Improving the Management of Instruction in 
Occupational Education in New York State . 



ERIC 
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design into operation by recording data and by providing 
analyses necessary to support decisions regarding classrv^om 
instruction, classroom resource allocation, student guidance 
and student placement. 

This section discusses the general concept of IS£S 
for occupational education, briefly reviews systems currently 
used in occupational education in New York State, draws some 
conclusions regarding the current situation, and based upon 
these conclusions, makes recommendations for future work. 

a . Concepts for Developing ISFS in Occupational 
Education 

A general goal that ISES for occupational educa 
tion shares with other processes within occupational education 
is to improve the effectiveness of occupational education pro- 
grams. ISES improves program effectiveness by providing a 
means for designing, implementing, evaluating, and monitoring 
programs to improve attainment of terminal, program objectives 
Associated with this goal, ISES possesses the following objec- 
tives : 

• To provide an explicit mechanism for formu- 
lating and evaluating terminal objectives 
associated with occupational education 
programs ; 

e To provide an explicit mechanism for design-- 
ing curricula in terms of their overall 
modular structure and the contents of 
individual modules; 

• To provide procedures for acquiring and 
processing data on teaching activities, 
instructional resources , and criterion- 
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referenced test scores, and for preparing 
reports based on these data to support 
local management of the delivery of instruc- 
tion; 

• To provide procedures for monitoring and 
evaluating programs and their component 
modules ; 

• To provide documentation of student perfor- 
mance for student placement and certification; 

• To provide students (and parents) with cur ent 
information regarding students * progress 
based on measurement of student progress 
towards intermediate performance objectives. 

The first two objectives are associated with 
program analysis and design; the remaining objectives are 
associated with information processing. 

1 . ) Steps in Program Analysis and Design 

Meeting the objectives associated with pro- 
gram analysis and design requires taking three steps. 

a . ) First Step: Defining Terminal Objec - 
tives 

ISES program analysis and design begins 
with identifying the skills required for entering specific 
occupations, or equivalently , defining terminal, program objec- 
tives. The specific set of skills required for a particular 
occupation comprises a set of terminal objectives for the 
educational program intended to train students for that occupa- 
tion. Some of the specific skills comprising the set for a 
given occupation are unique to that occupation; some are common 
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to a few, closely related occupations; some (reading or simple 
arithmetic skills, for example) are common to all occupations. 

Program design needs to be based on 

information concerning the requirements for obtaining jobs* 
2 6 

One way of obtaining information for the design of a program 
(i.e., its terminal objectives) is to obtain it from persons 
actively engaged in the occupation for which that program 
trains students. Thus program design elicits information from 
employers and employees (e.g., through their labor unions). 
For each program, a formal group might be established consisting 
of selected individuals (educators, employers, and union repre- 
sentatives) whose experience and capability are required to 
formulate terminal objectives. A consensus then can be reached 
on a provisional definition of terminal objectives for the 
program. 

Program design of this type generally 
proceeds most effectively by iteration. Using existing data, a 
design is adopted, and subsequent to implementation, data can be 
obtained to improve the design. As the refined design is imple- 
mented, more data become available and the design is refined 
further . 



"^^For example, an alternative approach is discussed in Part IV 
of the proposal for the development of Targeting and ISS in 
Occupational Education submitted by SED to the U.S. Office of 
Education. The alternative discussed in that proposal involves 
professional job analysts (to "cluster" jobs having commonalities) 
and professional occupational education program designers (to 
formulate terminal and intermediate objectives) . Job analysts 
and program designers obtain information from existing liter- 
ature or from persons having pertinent knowledge. 
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This process of iteration never 
completely ceases because job requirements generally can be 
expected to change with time. Thus, even if a set of objec- 
tives were adequate at a given point in time, cha;iges in job 
requirements subsequently might make the design obsolete. 

b . ) Second Step: Modularizing Programs 

Although the initial list of terminal 
objectives is obtained by consensus among various "^roups, the 
second step in program analysis and design primarily involves 
occupational educators. The second step consists of breaking 
down terminal objectives into sets of specific skills or inter- 
mediate objectives (sometimes called instructional, performance, 
or behavioral objectives) . This process is likely to be per- 
formed best by occupational educators. 

For the purpose of delivering instruc- 
tion within a program, the skills or intermediate objectives 
comprising the program are organized into units of instruction, 
called modules, in which closely related skills are taught 
using instructional resources that are related to each other. 
Modules within a program must be logically connected to each 
other, and must relate to the terminal objectives of the program. 

Figure 6 depicts the concept of a 
"branching tree of modules" for occupational education. At 
the base of the "tree," students enter a module in which 
skills necessary for all occupational programs are provided. 
The student progresses upward through the "tree" by taking 
modules that are increasingly more specialized as the "tips of 
the branches" are approached. By the time a specific "tip" is 
reached, students will have acquired the full set of skills 
necessary for entering a particular occupation. 
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c . ) Third Step: Establishing Standards 
and Measures 

The third step in the anlaysis and 
design process is development of standards and measures of 
^=^f fectiveness for program evaluation. Evaluation data indicate 
how well objectives are being achieved. Evaluation data are 
related to terminal (program) objectives and intermediate 
(module) objectives. Evaluation data provide a means of 
accountability, in terms of these objectives, at the SED, LEA, 
and classroom levels. 

Evaluation data pertain to effective- 
ness. In addition, data pertaining to costs are required. 
These might be expressed in terms of cost per student, cost per 
graduate, cost per initial placement, and cost per wage-earner 
over the long-term. 

To .monitor the delivery of instruc- 
tion and the mastery of skills, criterion-referenced measures 
of effectiveness must be defined for intermediate objectives. 
Skills check lists might be an instrument for entering data 
regarding skills mastery into a computer to record student pro- 
gress. In cases where skills are not acquired, the computer 
can provide teachers with information regarding available 
alternative instructional methods that might facilitate mastery. 

2 • ) Steps in Implementing Information Processing 

Implementing the information processing 
component of ISES can be based, to a considerable degree, upon 
procedures developed for use in the GuMderland Central School 
District to support the delivery of instruction in reading and 
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mathematics curricula. Software, hardware, and instruments 
currently used need to be modified to tailor them to the special 
features of occupational education. In general, these modifica- 
tions are straightforward and can proceed in parallel with 
program analysis and design. However, two new information 
processing procedures not required in the Guilderland Central 
School need to be developed for occupational education. 

Procedures developed for use in the Guilders- 
land schools do not include procedures for specifying the opti- 
mal point of entry into a program for a secondary student 
wishing to make a change of programs or for an adult with work 
experience wishing to acquire new skills. These procedures 
need to be developed. They would be used to assess skills 
possessed by the person entering a program and compare them 
with skills associated with the terminal objectives of that 
program. Based upon this information, the point of entry into 
the program would be determined so that modules are taken that 
provide only skills not yet acquired by the student. Such 
procedures require specifying the component skills of each 
program in a codified way so that the correspondence of skills 
taught in different occupational programs can be recognized. 

Procedures for scheduling also are required. 
Scheduling of modules having different lengths is quite diffi- 
cult. Computerized scheduling techniques are needed to permit 
flexibility in module design and to take full advantage of the 
potential for increasing cost-effectiveness. 



The Development of an Instructional Support System , renewal 
proposal, submitted to the Guilderland Central School District 
Number 2, Guilderland, New York, 1972. 
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In addition to providing information to be 
used directly by educators, the information handling component 
of ISES provides a means of monitoring the system. Administra- 
tors are given evidence that the system is operating according 
to specifications, i.e., that student plans are being formulated, 
that schedules are being devised,, that resources are being 
allocated, that students are being instructed, tested and passed 
from module to moduJ.e according to plan, that documentation is 
occuring, and that students are being placed in jobs matching 
their acquired skills. 

3 . ) Overview of the Advantages of Modularized 
Curricula 

A major advantage in modularizing curricula 
is the improved efficiency of utilizing educational resources 
that modularization makes possible. Modules defined for 
separate programs may contain many common skills. Where re- 
dundancy exists between modules of different programs, a single, 
common module can be defined that provides instruction for all 
programs to which it is applicable. (See Fig. 6, page 3 8.) 
Thus a single module might be utilized in programs having 
different terminal objectives. Resources necessary for deliv- 
ering instruction in different programs can be shared through 
the use of these resources in a module common to several pro- 
grams. Common use of resources leads to cost reductions. 

A second advantage in modularizing curricula 
is that students do not have to commit themselves to a parti- 
cular occupation prematurely, for example, they do not have to 
make an unalterable decision at the time of their entry into 
occupational education. Rather, they can proceed towards a 
preferable occupation with continual options of making choices 
regarding the occupation they wish as they elect increasingly 
more specialized modules. 
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*\ third advantage is that program changes 
near the end of a program (changes that might require the stu- 
dent to master a substantial number of skills within the newly 
elected program) can be undertaken in an efficient manner be- 
cause a mechanism is provided for explicitly identifying 
required skills. Guidance counselors and students readily can 
plan "re-tracking" in which only modules required to provide 
skills required by the newly elected occupation are taken by 
the student. In addition, individuals who wish to change 
occupations after having been employed for some time can profit 
frou the modular curriculum design. Such individuals are likely 
to possess skills acquired during their period of employment 
and to possess greater maturity, irotivation and responsibilities 
than secondary level students. Consequently, the optimal re- 
training program for such individuals can be determined by 
special tests and can utilize special modules that take advantage 
of these students' maturity and experience, and that minimize 
temporary losses in their earning power. 

4 . ) Summary of the ISES Concept 

The ISES concept briefly described above 
has the following three important characteristics: 

• Iterative methods are used for deter-^ 
mining terminal, program objectives 
(equivalent to job entry requirements) , 
and for determining intermediate instruc- 
tional objectives; 

o Methods are provided for managing pro- 
grams and instruction, and for iterative 
module design based on data; 
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# Iterative methods are provided for setting 
standards for and measuring student per- 
formance and program effectiveness. 

Attaining the goal of comprehensive instruc- 
tional management requires developing a system based on the 
concept that incorporates all three characteristics. 

b . Review of Agriculture Curriculum Modularization 
and ESCOE 

Within SED, two activities related to ISES cur- 
rently exist. These are the modularization of the agriculture 
curriculum and the development of the Evaluation Service Center 
for Occupational Education (ESCOE) . In this section, these 
two activities are briefly described and critiqued in terms of 
the ISES concepts discussed above. 

28 

1 . ) Agriculture and Curriculum Modularization 

An approach to curriculum modularization, 
compatible in some respects to the approach implicit in the 
ISES concepts discussed earlier, has been undertaken in the 
agriculture curriculum . 

This work began in 1967- and currently is 
being carried on under the direction of Mr. Lee Traver, Chief 
of the Bureau of Agricultural Education in the Office of 
Occupational Education at SED. Initial work consisted 



The discussion of this section is based upon a meeting with 
Mr. Lee Traver, Chief of the Bureau of Agricultural Education, 
and upon written materials provided to RRI by Mr. Traver. The 
purpose of this brief discussion is to provide an overall 
critique of the current approach and thus it considers only 
those characteristics deemed most relevant to that critique. 
Other published materials on agriculu'ire curriculum modulariza- 
tion describe the approach in detail. 
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primarily of partitioning the agriculture curriculum into five 
major areas.. This partitioning wan intended to provide a 
framework for defining programs of instruction in which closely 
related sets of skills are taught. Although the initial 
approach was not modularized as defined in the discussion of 
the ISES concept earlier in this report, it represented a first 
step towards modularizing an occupational curriculum. 

The second step v;as to revise the partition- 
ing of the agriculture programs to include seven (rather than 
five) major areas, to define modules more explicitly, and to 
conduct pilot studies. During the 1970-71 school year a pilot 
program was instituted in two BOCES. Encouraged by the results 
of the first pilot program, a second pilot program was under- 
taken during the 1971-72 school year in which nine BOCES and 
the Salem School District participated. However, subsequent 
to these two pilot programs, BOCES and school districts only 
have been informally using and evaluating the work* 

The curriculum center in SED has supported 
this work by providing funds (under VEA-68 ) to compensate 
educators participating in tlie design of modules. The Instruc- 
tional Materials Service at Cornell University also has 
suopcrted the work by providing lists of instructional materials, 
and helping to guide and advise educators involved in module 
design. 



29 

Vocational Education Amendments of 1968, Public Law 90-576. 
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a . ) Approach Used in Modularizing the 
Agriculture Curriculum 

The agriculture program has been div- 
ided into seven general areas termed "major groups." Within 
each major group, several occupational areas were defined, 
and, within each area, specific occupations were defined. 

Committees consisting of educators 
(teachers, guidance personnel, and administrators) and advisors 
(employers and union representatives) compiled sets of tenninal 
objectives for each specific occupation. The involvement of 
employers and workers active in the field was encouraged. 
However, their role was viewed as providing advice rather than 
actively participating in specifying terminal objectives. In 
practice, the involvement of noneducators ranged from active 
decision-making at one extreme to informal review at the other. 
The ISES concepts discussed earlier strongly advocate active 
participation of noneducators as one alternative method for 
selecting terminal objectives. 

After terminal objectives were estab- 
lished for agricultural occupations, a modularized curriculum 
was designed for each occupation. The first step was specifying 
modular units of instruction comprising the program for each 
job. The modules were constrained to accommodate thirty hours 
of instruction and to be largely independent or "free-standing" 
(in the sense that having one module as a prerequisite for 
another was avoided) to permit freedom in scheduling. Given 
these constraints, qualified educators designed modules in 
terms of a general description, units of instruction (topics) , 
objectives (intermediate or performance objectives) , content, 
teaching methods, student application activities, evaluation 
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procedures, and resource materials. The^^e terms have the 
following meanings: 

• Units of instruction (topics) are 
subdivisions of module content in 
instructional terms. (The current 
process calls for defining these 
units in advance of defining per- 
formance objectives; logically, 
however, the objectives ought to be 
defined prior to prescribing aspects 
of instruction intended to lead to 
attainment of objectives.) 

• Performance objectives are defini- 
tions of measurable capabilities 

the sLudent is to develop as a result 
of taking the module. (Defining 
these capabilities logically should 
be the first design effort in module 
design; however, in the current 
approach, objectives are defined 
after the units of instruction are 
specified. In the current process, 
objective definitions also state 
where are under what conditions the 
student is to acquire skills, and the 
level of mastery expected.) 

• Content refers to the explicit compo- 
nents of a unit of instruction. For 
example, in a module entitled "Feed- 
ing Dairy Cattlo," the units of 
instruction are "Digestive Tract and 
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Processes," "Available Feeds," 
"Nutritional Needs of Springing 
Heifers and Dry Cows," "Nutritional 
Needs of Milking Cows," and "Feed- 
ing Practices"; content elements 
for the first unit might be "anatomy" 
and "physiology-" (Although these 
are appropriately part of the module 
desiqn, they should be developed 
sys^tematically by basing them upon 
the objectives defined far the 
module. In the current method, 
dependence of content upon objectives 
is not clearly identified.) 

e Teaching methods, student application 
activity, and available resource 
materials are self-explanatory, closely 
related components of module design. 

• Evaluation procedures are methods to 
be used to determine student achieve- 
ment. A variety of "tests" have been 
prescribed for use in each module for 
measuring mastery of skills defined 
in the objectives. These are 
criterion-referenced tests; the objec- 
tives serve as the criteria. 

As students enter the agriculture pro- 
gram, a guidance counselor explains the modularized curriculum, 
the major groups, the occupational areas and the terminal objec- 
tives associated with each career. The student and counselor 
then formulate a training plan in terms of modules to be taken 
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and, when necessary, the sequence in which they are to be taken. 
Based upon the plans devised for all students, schedules are 
planned for the year; schedules are arranged to permit teaching 
those modules elected by the greatest number of students. Thus, 
the system is very sensitive to the needs of students 

An additional attribute of the method 
currently used is that the explicit definition of module con- 
tent in instructional terms makes clear to teachers the subject 
matter of the module; a teacher can recognize areas in which 
his background is strong and others in which it is weak. Thus 
a teacher can take steps to reinforce his capabilities in areas 
of relative weakness. 

b. ) Significant Discrepancies 

This section addresses the most 
significant discrepancies between the currently used approach 
and the proposed ISES concept. Attention is given to those 
characteristics that appear to be significant in terms of 
standards or criteria derived from the ISES concept. This 
brief critique is intended to provide a basis for specifying 
the first steps of iteration required to develop ISES from 
current efforts within SED. 

i . Module Length 

A significant constraint on the 
approach that has been used in agriculture is that modules are 
designed with a single length: thirty hours of instruction. 
The purpose of imposing the constraint is to minimize the dif- 
ficulty of scheduling. Scheduling fixed length modules is 
difficult; scheduling variable length modules appears to be 
impractical without computer assistance. Lacking cost-effective 
methods of scheduling variable length modules, work on the 
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agriculture curriculum had to proceed within the constraint 
of single-length modules. 

Using single-length modules im- 
poses a limit on flexibility in module design. A module that 
could be shorter than thirty hours currently might have to be 
"stretched" or combined with elements of a related module to 
form a single module of thirty hours duration. 

ii. Module Interdependence 

A consequence of the constraint 
of single-length modules is that modules are designed to be 
"free-standing" or independent. Such free-standing modules 
can be scheduled randomly because of minimal interdependence 
among modules. However, because modules are free-standing, 
improvements in program efficiency that could result from the 
sequenced, "branching tree" approach (described in the earlier 
sections discussing ISES concepts) may not be realized. 

iii. Module Evaluation and Redesign 

The lack of explicit mechanisms 
for evaluating and redesigning modules also represents a sig- 
nificant discrepancy betv;een the current approach taken in 
ESCOE and the ISES concept. At present, evaluation forms are 
provided to students and educators to record comments on the 
modular approach and on individual modules. These materials 
provide information that can support evaluation and redesign 
to some extent, but they are limited because they provide 
insufficient objective information. 

As stated in the ISES concept, 
measurement techniques are needed to evaluate the effectiveness 
of modularization, the validity of terminal objectives, the 
utility of intermediate objectives, and the effectiveness of 
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other components of modules. As in all system designs, measures 
of effectiveness should be related directly to objectives. 
Performance should be compared to objectives using measures 
related to the objectives; such objective-based measures need 
to be formulated for the modularization approach, each program, 
and each module. 

iv. Monitoring Mechanisms 

Explicit procedures for monitor- 
ing have not been developed in the current, agricultural 
modularization effort. According to the ISES concept, ways 
should be provided to verify that student applications are 
distributed, received, and processed; chat counselling is 
occurring; that individual student programs are being planned; 
that schedules are being devised; that resources (personnel, 
equipment, supplies, and facilities) are being allocated and 
utilized in delivering instruction; that students are attending, 
acquiring skills, and proceding from module to module; that 
work to formulate and revise terminal and intermediate objec- 
tives is being performed. 

V. Procedures for Formulating Ter - 
minal Objectives 

The final discrepancy is the 
manner in which terminal objectives have been formulated. 
According to the ISES concept, the selection of terminal objec- 
tives needs to be based on knowledge of the skills required in 
occupations. One way to obtain this knowledge is to formally 
involve noneducators active in the fie]ds for which students 
are being trained throughout the processes of defining terminal 
objectives. These noneducators have an intimate and thorough 
understanding of job requirements and can provide useful infor- 
mation and judgements pertinent to terminal objectives. Given 
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this information, educators can better understand requirements 
of employers and translate these requirements into suitable 
terminal and intermediate, program objectives. 

2.) ESCOE ^^ 

The Evaluation Service Center for Occupa- 
tional Education (ESCOE) has been developed with the intent 
of improving the efficiency of delivering occupational instruc- 
tion at the local level. Work in ESCOE currently emphasizes 
the creation of banks of objectives for courses of instruction. 

a . ) Overall Approach 

ESCOE is intended to serve as an aid 
to teachers and as an accountability instrument. It is intended 
to perform its functions flexibly, to accommodate wide varia- 
tion in local usage, and to evolve iteratively towards an 
optimal system. 



Of) 

The discussion of this section is based upon a meeting with 
Dr. Louis Cohen, Chief of the Bureau of Occupational Education 
Research, and upon the following written materials provided to 
RRI by Dr. Cohen: Massachusetts and New York Evaluation Service 
Center for Occupational Education (planning document; published 
May, 1970) ; Evaluation Service Center for Occupational Education 
(final report submitted to Massachusetts and New York Depart- 
ments of Education by the University of Massachusetts Center 
for Occupational Education; June, 1972); The Compatability of 
ESCOE with Other New York State Education Department Instruc - 
tion, Resource and Information Systems (report prepared by the 
Institute for Educational Development in cooperation with the 
Ne,7 York State Education DeparLment Bureau of Occupational 
Education Research; November, 1972) . 

The purpose of this brief discussion in this section is to 
provide an overall critique of the current approach, and thus 
it considers only those characteristics of the approach deemed 
most relevant to that critique. The published materials cited 
above describe ESCOE in detail. 
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To realize these intentions, the 
approach used in ESCOE has been to elicit from educators (teach- 
ers and administrators) suggested objectives, tests, and re- 
sources for courses of instruction. Primary emphasis to date 
has been placed on developing a bank of objectives; secondary 
emphasis has been placed on developing a bank of test items 
associated with objectives; tertiary emphasis has been placed 
on developing a bank of instructional resources (models, film- 
strips, texts, slide sets, etc.) associated with the objectives. 
However, substantial effort has been devoted only to ojective 
bank development. 

Bank development proceeds by asking 
educators to submit descriptions of suggested, useful objec- 
tives, tests, and resources to ESCOE. 

Educators are asked to define the 
overall objective of a course, describe the overall topic com- 
prising the course, list and define subtopics, and finally 
formulate specific subob jectives associated with each subtopic. 
(Presumably, these subob jectives are equivalent to intermediate 
objectives described in the earlier section discussing the ISES 
concept.) At the same time, educators provide lists of tests 
and resources appropriate for each subob jective. Information 
acquired in this manner is assigned a code number beginning 
with a number corresponding to a U.S. Office of Education 
occupational program code number and ending with six digits 
that identify the topic and subtopics of the course. 

After code assignment, the information 
provided by educators is entered into a computer file. A sub- 
sequent request for information regarding objectives, tests, 
or resources is translated into codes for the appropriate topics 
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and subtopics comprising the request. The computer is inter- 
rogated and printouts are generated that contain all informa- 
tion in the files (i.e., lists of subob jectives , tests, or 
resources) pertinent to the request • 

In addition to the Bureau of Occupa- 
tional Education Research, other education-related offices and 
bureaus v/ithin the State currently are involved in ESCOE opera- 
tions. For example, the Educational Programs and Studies 
Information Service (EPSIS) typically receives requests for 
infomation regarding objectives, tests, or resources, and 
forwards these requests to the Office of Research and Evalua- 
tion. The Special Education Instructional Materials Center 
(SEIMC) has participated in cataloging resources relevant to 
specific subobjectives. The computer of the Computer Based 
Research Unit (CBRU) system, housed within the Curriculum and 
Evaluation Center of the State University College of Buffalo, 
has been used to store and retrieve ESCOE information. 

b. ) Significant Discrepancies 

Several discrepancies appear to exist 
'between the approach taken in ESCOE and the ISES concept dis- 
cussed earlier. Discrepancies discussed below do not represent 
deficiencies in an operating system (i.e., deficiencies that 
might be eliminated in time through iteration) but discrepan- 
cies between ESCOE and ISES on the conceptual level. This dis- 
cussion recognizes that although ESCOE is a development effort, 
pursued largely on an informal basis, the existing banks of 
information within ESCOE (particularly objective banks) repre- 
sent a valuable resource for ISES program analyses and design. 

i. Program View; Terminal Objectives 

In ESCOE, a view of instructional 
programs tailored to achieve specific terminal objectives is 
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not taken. Instruction is viewed as consisting of a set of 
individual, independent courses, and objectives are defined 
for courses v/ithout reference to integrated programs or ter- 
minal objectives. Consequently, potential improvements in 
the efficiency of delivering instructions may not be realized. 
The ISES "branching*' tree of modules and its attendant bene- 
fits are precluded because of the absence of a program-oriented 
perspective. 

ii , Procedures for Fomulating Objec- 
tives 

As well as lacking 'terminal, 
program objectives, the ESCOE approach relies soley upon educa- 
tors to formulate course objectives. No explicit mechanism 
exists for providing zhe educators who are responsible for 
objective formulation with information on job market require- 
ments. As discussed earlier in the section on agriculture 
curriculum modularisation, one way to obtain information 
necessary for formulating terminal objectives is to obtain it 
from employers and organizations of employees active in the 
field for which objectives are being formulated. 

iii. Evaluation Procedures 

Evaluation mechanism designs 
appear to be absent in ESCOE. Evaluation methods only appear 
as tests to be used in conjunction with subob jectives within 
each course. Presumably, ttiSts specified in associat-^on with 
subob jectives might provide a basis for evaluating course effec 
tiveness; however, little attention appears to have been given 
to test development and validation. 
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iv. Monitoring Mechanisms 

No design has been developed for 
monitoring instruction and no attention has been given to the 
need for reports to teachers or for recording of student achieve- 
ment or objectives. Consequently, no mechanism for monitoring 
is apparent and no hardware or software have been specified to 
support the monitoring process. 

c . ) D evelopment Plan 

A programmatic plan for developing 
and implementing ISES needs to be formulated. The plan should 
take into account the attributes and shortcomings of existing 
work in agriculture curriculum modularization and ESCOE, and 
should permit building from the strengths of these efforts 
toward an integrated instructional support and evaluation sys- 
tem that can serve the needs of occupational education. 

i . Elements of the Plan 

A significant shortcoming of the 
current efforts is that they are uncoordinated. They are pur- 
sued as independent ventures. Neither gains from the other, 
and no way exists to anticipate and avoid duplication of effort 
and cost. The two efforts could be integrated by establishing 
a single entity to coordinate them and to promote efficient and 
effective progress towards further defining and realizing the 
ISES concept. 

Effort might be focused on one 
or a very small number of programs. Because of the work already 
performed in the agriculture program, and work in ESCOE on the 
practical nursing program, these two programs are obvious 
possible caiididates. 
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As stated earlier in the dis- 
cussion of ISES concepts, design proceeds from the formulation 
of terminal objectives, to the translation of terminal objec- 
tives into intermediate objectives • The set of intermediate 
objectives then is partitioned into subsets of objectives for 
which related resources and instructional methods are needed. 
These subsets of related intermediate objectives are used to 
define modules, and the dependencies of one subset (or module) 
upon acquisition of skills associated with other subsets (or 
modules) defines a sequence in which modules are to be taken. 

Using the agriculture program as 
an example, work already performed can provide a starting 
point for defining terminal objectives. Individuals involved 
in agriculture could be involved in reviewing and approving 
tentative terminal objectives and in contributing alternatives 
to those already formulated, modifying those deemed suitable, 
and suggesting new ones. (However, in the future, formulation 
of terminal objectives might actively involve noneducators 
from the outset.) 

ESCOE banks of agriculture-related 
obje::tives could be examined to identify candidate intermediate 
objectives for each terminal objective. The existing coding 
system for each ESCOE intermediate objective might be suitable 
for sorting existing objectives; if the existing coding system 
is inadequate for sorting purposes, an alternative might be 
devised . 

Concurrent with design effort 
directed towards program objectives, module objectives, module 
contents and module interrelationships, design effort also must 
be directed towards measures of effectiveness for evaluating 
the modularization approach, programs and modules* Having 
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defined the measures, procedures for obtaining and evaluating 
data must be developed, and formal procedures for redesign 
must be established. 

Monitoring processes also must 
be designed so that a mean? exists to ascertain that actions 
intended to be taken in an operating system actually are taken. 

Software, remote data entry hard- 
ware, data entry instruments (e.g., test score sheets), and 
reporting instruments (e.g., skills mastery reports and module 
completion reports) also must be specified and designed. 

To take full advantage of the 
potential benefits of ISES, a mechanism for scheduling variable 
length modules needs to be developed. Such a scheduling method 
eliminates the constraint of fixed-length modules. At this 
juncture, program analyses and design faces a "trade-off" 
decision. The advantages to be gained by developing a schedul- 
ing mechanism may not outweigh those to be gained by proceding 
with implementation of fixed-length modules. A preliminary 
investigation may be required to make the decision; for example, 
existing scheduling "packages" miyht be examined. A variety 
of scheduling "packages" currently are available. One of these 
may be suitable or adaptable, or design of a new "package" 
may be required. 

Formal evaluation and redesign 
processes, formal monitoring processes, software, hardware, 
instruments, and scheduling methods would be applicable to all 
programs. They are required whether a single program is modu- 
larized or all programs are modularized. Thus they must be 
developed even though the cost may be high, in order to proceed 
with establishing ISES concepts for the programs in which initial 



F/290-5-00 



- 57 - 




64 



work takes place. As additional programs become modularized 
in the future, the costs for developing these processes and 
methods become distributed among all the programs and the "per 
program" costs become lower. 

ii. Sximmary 

Figure 7 depicts the develop- 
mental process discussed above, 

3 . Attendance Subsystem 

This section discusses the potential of incorporating 
attendance procedures into RESOE. 

RESOE is based on a unit-iile record-keeping system. 
A file is established for each student and updated periodically. 
Attendance systems have equivalent characteristics. Integrating 
required attendance operations with RESOE operations may stream- 
line both operations by avoiding duplication of effort. Com- 
bining the two operations can proceed from either of two 
perspectives: currently used attendance procedures for file 
creation and updating might replace corresponding, currently 
used RESOE procedures; RESOE might be expanded to include 
attendance operations. 

Integrating existing attendance and RESOE operations 
potentially leads to two benefits. Expensive and time-consuming 
"manual" procedures now commonly used (particularly in small 
local agencies) for collecting and processing attendance data 
can be replaced by a more efficient, rapid, computerized pro- 
cedure; RESOE student files can be revised continuously and 
automatically • 

a. Current Situation 

In this section, the current situation is described 
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in two parts* First, local uses of attendance data briefly 
are reviewed. Second, an overview is provided of current 
methods of acquiring and reporting attendance data, 

1. ) Uses of Attendance Data 

Local agencies are required to obtain and 

31 

record attendance data for the following general purposes : 

• To submit Weighted Average Daily Atten- 
dance (WADA) reports in order ^.o 
receive State Aid funds; 

• To obtain data necessary to perform 
mandated student supervision and 
counselling services . 

State Aid funds are apportioned among local 
agencies by using a formula involving attendance data as the 
critical variable. Daily attendance data are compiled into 
reports at standard intervals (four-v/eek or five-week) • An 
official record of daily attendance must be maintained. Tradi- 
tionally, the class register has been used as this legal record, 
but r-oniputer files now may be used. In the latter case, a 
computer printout becomes the official document. 

Local agencies are required to maintain 
cognizance of individual student attendance so that, when 
necessary, they can provide counselling for students or take 
legal action against parents. One or morp. members of the local 
agency staff are designated attendance officer. Persons having 



•'•^See "Legal Information Concerning School Attendance/" a 
report prepared by the SED Bureau of School Social Services, 
revised September, 1970. 
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the responsibilities of attendance officer rely upon atten- 
dance data and other informatic.i to take action (counselling 
or legal) in cases of illegal absence, and to investigate 
suspicious absences or patterns in absences to ascertain whether 
counselling or legal action are in fact required. (In addition, 
attendance officers typically assume responsibilities associated 
with maintaining a census of school-age children in the school 

district, and maintaining records of student employment, issu- 

32 

mg work permits, etc. ) 

To support attendance officers in local 
schools, most BOCES provide attendance reports to local schools 
every day. These reports provide attendance data or; students 
attending the BOCES from each local school affiliated with the 
BOCES. In a typical BOCES, each day consists of a morning and 
afternoon session, and reports are prepared for local schools 
at the end of each session. 

b. Current Methods of Obtaining and Reporting 
Attendance Data 

This section briefly describes two currently 
used procedures for collecting and processing attendance data. 
This description is based on information provided by personnel 
associated with a small number of BOCES in Regions 8, 9, 10, 
and 11, and with the City of Rochester School District. The 
situation existing in these BOCES seems to be typical of small- 



See "Legal Information Concerning School Attendance,*' a 
report prepared oy the SED Bureau of School Social Services, 
revised September, 1970. 
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and medium- sized BOCES throughout the State. The situation 
in the City of Rochester is an example of a modern, efficient, 
computerized individual student file, record-keeping system 
that obtains and processes a variety of data, including atten- 
dance data, for a variety of reporting and analytic purposes. 
(The City of Syracuse and the large Nassau County BOCES use 
similar systems.) 

1 . ) The Situation in BOCES 

BOCES attendance procedures rely heavily 
upon manual effort. An occupational education teacher in a 
BOCES typically provides data each day on students attending 
his class by filling out a form or a computer card. The form 
(for an entire class) or computer card (for ee ::h student) then 
is transported to a central office v/ithin the BOCES. 

If a form is used, a secretary or clerk 
(in one BOCES, a nurse) transfers data from each class to a 
central log for the BOCES and a daily attendance report for 
each affiliated sch'^ol . Information may be transferred to the 
school verbally over the telephone, or by carrying a report 
on the bus that returns students to their home school. 

If a computer card is used, cards are 
sorted by machine into groups for each school. The sorted 
cards are then manually examined to enter information into the 
BOCES log and prepare daily reports for each school. 

Data on students attending BOCES are sub- 
mitted to affiliated schools each day so that attendance 
officers in each school can be appraised of their students' 
attendance in a timely fashion. 
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In many cases, school and BOCES attendance 
data are processed by a data-processing BOCES to prepare 
standard (four-week or five-week) attendance reports in compli- 
ance with "State Aid" requirements. 

2.) The Situation in the City of Rochester 
School System 

The Rochester school system is using a 
unit-file system of student record-keeping that is analogous 
in many respects to RESOE. (As stated earlier, the City of 
Syracuse and the Nassau County BOCES use similar systems.) 
A file is established for each student upon entry into a 
Rochester school. The student is identified by a seven-diait 
number that is retained throughout the student's enrollment. 
(A studenr returning to a Rochester school after transferring 
out of the system is reassigned his original identification 
number.) The file for each student contains his location in 
the school system, personal information, grades, attendance, 
and other data. The file can be expanded (or contracted) as 
new needs for data are perceived (or existing ones are deemed 
obsolete) . 

Prior to the beginning of each school year, 
schedules and class assignments are made for each student. 
The system is sufficiently flexible to accommodate changes that 
may occur subsequent to the opening of school. (Typically, 
class assignments for about 25% of the students must be changed 
after school opens.) 

In each school, an attendance register is 
kept for each instructional period. This is an official doc- 
ument that never leaves the school. Teachers enter attendance 
data in the register during each period. At the end of the 
five-week attendance period, a secretary in the school office 
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transcribes attendance data from the register to a form that 
is readable by a mark-sense optical scanner. (Rochester uses 
eight five-week attendance periods that coincide with its grad- 
ing periods. ) 

Attendance data are entered into individ- 
ual student files in the computer using an optical scanner. 
Periodic attendance reports then are generated from these data 
in compliance with "State Aid" requirements. In addition, and 
even more important to educators in the system, attendance data, 
as well as other data items in individual student files, rou- 
tinely are incorporated into reports specified by school admini- 
strators in Rochester. 

3. ) Summary of the BQCCS and Rochester Procedures 

This cursory examination of BOCES procedures 
associated with acquiring and processing attendance data indi- 
cates that the manual procedures being used in small BOCES and 
satisfying local needs are adequate for preparing WADA reports. 
However, a centralized, computerized procedure that is less 
costly than current, labor-consumptive methods might be accept- 
able to BOCES. 

The procedures used in the City of Rochester 
school system employ modern, efficient, computerized methods. 
Much of the experience gained in Rochester might be applied to 
developing a RESOE-related attendance package. 

Recalling the two perspectives mentioned at 
the beginning of this discussion, the perspective in which 
attendance procedures are added to RESOE is applicable to the 
small BOCES, and the perspective in v/hich RESOE relies upon 
existing attendance procedures for RESOE file creation and up- 
dating is applicable to the City of Rochester and equivalent 
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facilities* In the former case, small BOCES would gain from 
the replacement of labor-intensive procedures with RESOE-based, 
compu'-erized procedures for obtaining and processing attendance 
data. In the latter case, RESOE and personnel in the schools 
gain by utilizing existing procedures to support RESOE and 
therefore to reduce duplicative effort • 

c. Design Specifications 

Design specifications for a RESOE attendance 
package can be drawn from two views* In one view, specifica- 
tions are based on the requirements of local agencies. In the 
other view, specifications are based on the needs of RESOE for 
a mechanism to update enrollment files. 

1. ) Si^^ecif ications Based on Local Agency Needs 

The RESOE attendance package must be able 
zo support local WADA report preparation and student supervision 
and counselling. To do this, the package must perform the 
following functions : 

o Daily acquisition of attendance data; 

e Periodic processing of attendance data 
to provide standard individual student 
attendance reports for use by atten- 
dance officers within local agencies. 
(In the case of BOCES, reports to local 
schools must be generated daily) ; 

» Periodic (four-week or five-week) and 
yearly processing of attendance data to 
provide standard periodic WADA reports 
for obtaining State Aid funds; 
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^ Periodic data analyses for local atten- 
dance officers to reveal patterns 
(temporal or " inters tudent") in absences 
in order to detect individual student 
problems that may require counselling. 

2. ) Specifications Based on the Needs of RESQE 

The general need of RESOE is for a simple, 
cost-effective method to keep enrollment files current. An 
attendance package seems to be ideal for satisfying this need 
because local needs for improved methods of acquiring and 
processing attendance data can be satisfied simultaneously with 
satisfying the needs of RESOE* 

To satisfy the needs of RESOE, combining 
existing RESOE procedures with attendance procedures, must 
contribute to performing the following general functions: 

' o Establishing individual student files • 

e Updating student files by obtaining 
and processing data related to changes 
in student status within a local agency; 

9 Providing information for evaluation 
(of programs, modules, or instructors) 
by obtaining and processing attendance 
for inferring student "motivation". 

The last two functions depend upon data obtained directly from 
a classroom. 

Data related to student status include, for 
example, student transfers from one class to another within a 
local agency. (Another type of data that might be obtained 
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with attendance data from the classroom is data on student 

3 3 

skill masLery and module completion in an ISES context 
However, changes such as program changes within a local agency, 
and transfers into or out of a local agency might not be 
indicated by attendance; such data might be obtained directly 
from the administrative personnel of the local agency. 

Attendance data might be used to infer stu- 
dent "motivation" for the purpose of evaluating programs, 
modules (in an ISES context) or instructors. Developing reli- 
able measures for this purpose will be difficult, but atten- 
dance data, if properly used can contribute to evaluation. 
For example, unusual absences, (numbers or patterns) currently 
are used as an indicator of personal problems in individual 
students. Generally low student attendance in a local agency 
might indicate that factors outside the control of the agency 
are limiting the effectiveness of the programs in that agency. 
Generally low student attendance in a program might indicate 
inadequacies in program design. Generally low attendance in 
a particular module might indicate a deficiency in module de- 
sign or in the delivery of instruction. These are only a few 
obvious examples. Any attempt to use attendance data as a 
component in a measure of effectiveness must be based on very 
careful design and analysis of the measure itself. 

d. Generalized System 

Based on the requirements sketched above, the 
general nature of one possible way to combine RESOE and atten- 
dance procedures can be described. This description is intended 
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See Chapter III, Section B, Part 2 of this report. 
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only to illustrate the features implied by the requirements 
and specifications presented earlier. 

1 . ) Subsystem Exampl e 

At the time of enrollment in a program 
(at the beginning of or during the school year), each student 
fills out an enrollment form equivalent to the currently used 
RESOE enrollment form. This form acquires all data necessary 
for initially creating student files for enrolled students. 
An identification number for each student is established at 
this time. 

Based on the information obtained on enroll- 
ment forms, attendance-taking instruments (cards or forms) are 
prepared for each class. The instrument must obtain data on 
absences (legal and illegal) and tardiness. (Other data for 
file maintenance might also be obtained in the classroom, such 
as data on "dropouts," transfers, course changes, etc.) 

In the case of BOCES, data provided by 
daily classroom attendance instruments must be processed each 
day to provide attendance data* This capability is essential 
because BOCES must submit daily reports to the attendance 
personnel of the schools they serve. 

Periodic reports giving individual atten- 
dance data and summarized attendance data must be prepared. 
The scheduling of these reports is determined by WADA reporting 
requirements and local usage. RESOE will permit specifying 
unique reports at local user discretion and cost; however, to 
the extent possible, standard report formats might be designed 
to minimize costs associated with time expended specifying 
reports via a terminal. 
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A capability for analyzing patterns in 
absences can be useful to attendance personnel for detecting 
symptoms of personal student problems. If a student is absent 
to an unusual degree, appears to be regularly absent (or tardy) 
on a aiven day of the week or month, or if two or more stu- 
dents appear to be regularly absent together, these patterns 
may signify that a problem needing attention exists. (In 
addition, if modularized curricula are used, patterns asso- 
ciated with individual modules may provide information for 
evaluating modules or teachers to the extent that inferences 
can be drawn from student attendance in a module.) 

Attendance cards might be used to obtain 
a variety of additional types of information necessary for 
keeping secondary level student files current. For example, 
as the end of the school year nears, columns might be added 
to the attendance card in which the teacher can indicate that 
a student is expected to graduate* 

2 . ) Summary 

Figure 8 depicts the process encompassing 
relationships among the elements discussed above. Enrollment 
forms are filled out at the start of the process. Enrollment 
data provides the basis for preparing attendance cards. (As 
an option, in an ISES context, skills mastery cards are also 
prepared.) Attendance card data leads to updated enrollment 
files, attendance reports and pattern analyses. (The optional 
skills-mastery cards provide output in the form of reports.) 
Combined with information derived from sources other than 
attendance cards, follow--up questionnaires, enrollment forms 
(for the subsequent year or for updating during a given school 
year), comprehensive studenL record reports can be prepared. 
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e. Development of the Attendance Subsystem 



This discussion of attendance is based upon a 
preliminary investigation of a few existing systems and upon 
preliminary conclusions drawn regarding requirements. The in- 
vestigation is incomplete in the sense that information has 
been obtained from individuals in a small number of BOCES, and 
from a small set of documents describing attendance require- 
ments. However, further work should be undertaken to insure 
that the information obtained to date is complete, and that 
it adequately reflects the general case. 

Subsequent to confirming that information is 
complete and representative, current design objectives and 
specifications can be detailed further if necessary. 

Existing systems and procedures, including those 
independently developed and operated by local agencies and 
those purchased as proprietary packages, can then be investi- 
gated in terms of their ability to meet the objectives and in 
terms of the costs associated v/ith their use. An existing 
system might currently meet all the requirements; an existing 
system might be amendable to adaptation into one meeting all 
the requirements at a modest cost for redesign; an extensive, 
complete design effort may be required. 

Based on this analysis, an optimal system can 
be selected or designed, and implemented on a pilot basis. 

A pilot study might be performed involving one 
or a few BOCES, and perhaps one city school system. The pilot 
study verifies assumptions ma.de during design and analysis. 
It involves a limited commitment of resources and can provide 
invaluable experience for redesign and improvement prior to 
making a substantial investment . 
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The use of a remote terminal for entering new or 
corrected data, for requesting reports, and for specifying 
data analyses seems to possess considerable merit. The general 
affect of using remote terminals is to shift responsibility 
and activity for entering correct data toward the local user, 
and therefore to distribute responsibility and activity over 
the population of users rather than concentrating it at a 
central site. The general advantage of this situation is that 
the local user, who can best detect errors in data, enters 
edited data and makes corrections efficiently. In addition, 
the local user, who best understands his own needs for data or 
analyses, can specify requirements for reports or analyses 
directly to the computer. The general disadvantages of this 
situation are that users may not have required skills for 
effectively interacting with the computer, and they may not 
have the time or resources to devote to entering large amounts 
of data. 

In anticipation of the general need to address this 
topic in a systematic manner and the particular need to assess 
the feasibility of data entry and editing at a local site, a 
remote terminal was installed in a BOCES (the Charles G. May 
Occupational Center, in Mount Morris, New York) this fall as 
a "demonstration." The demonstration and some observations 
regarding the use of remote terminals are discussed below. 

a. General Description of the Demonstration Project 

The ajor unknowns of interest in the demonstra- 
tion involved huma. actors such as performance operating the 
terminal, and the feasibility of transferring data entry, 
editing and error correcting responsibilities to local agency 
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personnel. (Detecting and correcting errors in RESOE enroll- 
ment forms has consumed considerable RRI resources during the 
past two years. This situation is unavoidable in a procedure 
requiring that central facility personnel not familiar with 
local circumstances edit data.) Consequently, a very versatile, 
sophisticated terminal was used, a Ilazeltine model 2000 with a 
cathode ray tube (TV-screen) display, a printer, and a dual 
cassette recorder. 

In the demonstration, the terminal was to be 
used to edit and enter enrollment data into computer files. 
This application is quite repetitive and routine, and it in- 
volves entry of a large amount of data. The Hazeltine was 
selected for the demonstration to provide the option of inves- 
tigating more sophisticated applications if the opportunity 
appeared; it was not selected because it is the optimal unit 
for the primary application in this demonstration. 

Two methods of data entry were used. In the 
first method, termed "on-line, interactive," the terminal was 
connected to the computer using conventional (voice-carrying) 
telephone lines throughout the entire period during which data 
were entered. In the second method, termed "off-line, record- 
ing/' data were entered into the terminal memory, edited and 
then transferred onto a tape in a cassette recorder attached 
to the terminal . Subsequent to "filling" cassettes , the 
cassettes were mailed to RRI for entering data into the computer. 
(Alternatively, data on the tape might have been transmitted 
over telephone lines, in a single, relatively brief period.) 

In an introductory session at the BOCES, the 
terminal operator was instructed on the first method, and was 
observed during entry of data from a few sample forms. Sub- 
sequent to this introductory session, the operator entered data 
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daily, and data entry was monitored at RRI. Data were entered 
at a rate of approximately 50 forms per hour. 

After a period of several weeks using i-he first 
method, a cassette recorder was attached to the terminal, and 
the operator was instructed on the second method. Using this 
method, the rate of data entry was approximately 70 forms per 
hour. The rate increase resulted from having a display of a 
Llank enrollment form on the screen of the terminal during 
data entry. Information for this display was prerecorded on 
one of the two cassettes in the recorder. The operator simply 
used the teiminal as a typewriter, in effect, typing informa- 
tion on the displayed form. During typing, data were recorded 
in the memory of the terminal itself. While in the terminal 
memory, the operator examined displayed data for errors, 
corrected errors ^ and transferred data in the memory to the 
tape in the cassette. After completing entry of data from one 
form, displayed data were erased from the screen leaving the 
blank form displayed. Data from a subsequent enrollment form 
then were entered onto the displayed blank form, 

b. Assessment of the Demonstration 

The demonstration can be assessed on two grounds 
human performance and technical considerations. In both cases, 
the demonstration confirmed expectations. 

In general, data entered were ey:ellent in 
comparison to data entered from forms at the ce'^tral facility 
(RRI) , Thus, transferring editing (error detection and correc- 
tion, ambiguity detection and resolution) responsibility to 
local agencies appears to be effective. 
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1. ) On-line, Interactive Method 

In the first method (on-line, interactive), 
human performance was limited by the xack of a display of the 
blank form. This limit was severe in this demonstration because 
enrol "'.ment forms were designed to facilitate data entry by key- 
punching. Software could be developed to permit entry of each 
data field on the form into a separate line on the terminal 
display; such software would reduce this problem. Alternatively, 
a form could be designed to facilitate data entry using a 
remote terminal. (However; in the present case, the signifi- 
cance of this limitation is reflected by the increased rate 
of data entry from 50 forms per hour without a blank form on 
the screen to 70 forms per hour with a blank form on the screen. ) 

Two serious technical problems were assoc- 
iated with the on-line, interactive method; both involved the 
use of telephone lines. 

The first problem involving the use of 
telephone lines is that the lines connecting the Mount Morris 
Center to New York City were unreliable. For example, during 
tne first day of data entry, line noise and accidental dis- 
connections at the exchange in Rochester made data entry 
impossible. The major concern is that line noise can be inter- 
preted by the computer as a command from the terminal; an 
inappropriate command can seriously disrupt d^"^^ entry and make 
considerable data (thought to be properly entered) useless. 
The operator is compelled to check terminal status frequently 
in order to detect such perturbations as promptly as possible. 

The second problem related to using tele- 
phone lines is that the terminal is connected to the computer 
continuously during the period in which the operator is working. 
This leads to high telephone charges; for example, tha direct 
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dial rate to New York City from Mount Morris is $0.4 2 per 
minute, (Thus, assuming 50 forms are entered per hour and 
1500 forms are required for an agency, the charges would be 
$756.00.) 

In addition to problems associated v/ith 
using telephone lines, another technical problem exists ii* the 
case of on-line, interactive data entry. In the demonstration, 
a processor (a particular type of software package) called 
EDIT was used to create the files. EDIT is designed to facil- 
itate writing computer programs using a remote terminal, and 
it is not designed for creating files. However, EDIT is 
probably as suitable as any available processor for creating 
files and entering data via a terminal. More suitable pro- 
cessors for creating files currently are not available because 
other methods (e.g., keypunching or optical scanning) are 
almost universally used for such functions; these methods 
generally are far less expensive than using a remote terminal 
for high volume, repetitive entry of data into files. However, 
a processor tailored to large volume repetitive data entry and 
file creation could be developed. 

2. ) Off-line, Recording 

The second method (off-line, recording) is 
superior to the first in terms of human performance and tech- 
nical factors. 

The Hazeltine terminal's ability to display 
a blank form proved to be a significant asset from the perspec- 
tive of human performance. From this perspective, if large 
volumes of data must be entered via a terminal, the off-line 
recording method may be the optimal method. 
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Once data are recorded on the cassette 
tape (i.e./ transferred from the terminal's "memory" to the 
tape) / error correction is difficult • However, v/hile data 
are still in the terminal's memory, corrections are made quite 
easily, more easily than in the on-line, interactive method 
using the EDIT processor. 

Each cassette holds a given number of 
records. In the case of this year's enrollment forms, each 
side of the tape holds 300 forms. However, the tape does not 
provide an indication that the end of the tape is being reached, 
e.g., that one more form can be accommodated* In addition, 
tl. ; operator must enter an "end-of-tape" record as the last 
entry on each side of the tape. 

In the Hazel tine unit, a given record is 
difficult to find once it is recorded in the tape because the 
tape can only be rewound to its beginning. 

Turning off the recorded at the end of a 
data entry session sometimes causes spurious signals to be 
entered onto the tape. 

c . General Conclusions and Recommendations 

Based on the observations discussed above, some 
general conclusions can be drawn regarding the use of remote 
terminals for data entry and suggestions can be made regarding 
alternative methods of entering large volumes of data. 

As stated earlier, data entered via the terminal 
appear to be quite low in errors compared to data entered by 
conventional means. Thus a tentative general conclusion can 
be drawn that transferring to local agencies the responsibility 
for providing *'clean*' data is feasible* However, at 
present analysis of costs of entering data using remote 
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terminals versus costs for entering data by other means has 
not been performed. Such an analysis is desirable to justify 
the use of remote terminals for data entry. The justification 
is increased as the functions (for example, entering atten- 
dance, facilities or personnel data, or specifying reports) for 
which the terminal is used increase. 

1 . ) Conclusions Drawn from the Demonstration 

A remote terminal, particularly the 
Hazel tine model, is designed for interactive communications 
with a computer • For example, it is well suited for editing, 
correcting or updating files or for entering programs, spec- 
ifying reports, or performing data analyses. Large-volume, 
repetitive data entry is not a cost-effective application for 
such remote terminals • In the case where large amounts of 
data, e.g., enrollment data from several agencies, are to be 
entered via a single terminal, the terminal might act as a 
severe "bottleneck". If -zime is a factor, the "bottleneck" 
would be a significant problem^ 

If a remote terminal v/ere to be used for 
large-volume data entry, the off-line, recording method seems 
the most suitable. Technical difficulties associated with it 
might oe overcome by modest changes in hardware and perhaps 
by developing software for use at the central site or the 
terminal itself for detecting spurious errors or keeping a 
tally of records entered-. 

2, ) Alternatives and Recommendations 

A cost-effective system for remote da*"a 
entry should have the following characteristics: 
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• Hardware should involve simple 
technology, i.e., it should be in- 
expensive to purchase, easy to repair 
and usable by semi-skilled persons. 

© Recorded data should be easily cor- 
rected* As a corrollary, individual 
records should be easy to recall for 
examination^ 

« "Hard copy" should be generated for 
local editing and "receipting" pur- 
poses • 

• A rapia method should exist for enter- 
ing data into the central computer 
files and confirming that no errors 
have been introduced during the entry 
process , 

Remote data entry via a terminal that is 
eminently suited to flexible interactive operations (such as 
the Hazeltine) does not possess these traits* However , the 
following alternatives are v^?orth consideration: 

e Key-punch, key-tape or key-disc; 

9 Optical scanning (mark-sense) ; 

a Optical character recognition • 

Key-punch, key-tape and key-disc methods 
are inexpensive and conunoniy used for entering large volumes 
of data* To date, RESOE has relied upon key-punching, in 
which data are entered by key-punch operc tors onto computer 
cards; cards in turn are used to transfer date to computer 
tape or disc for storage processing operations * In key-tape 
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and key-disc, the key-punching of cards is eliminated; the 
operator inserts (key-punches) data directly onto tape or 
disc. (Some key-tape data entry v/as used this year in RESOE.) 

Optical scanning (mark-sense) is a very 
inexpensive, simple, and reliable method of reading forms and 
entering data by machine. However, the machine only responds 
to simple marks; the machine cannot interpret shapes. In 
situations where written information, such as names or 
addresses, are to be entered, the optical scanning form may 
have to be large and entering data on the form may be tedious 
and subject to error. 

Optical character reccgriition is u sophis- 
ticated, relatively expensive, but reliable, method of reading 
forms and entering data by machine. Optical character recogni- 
tion machine recognizes symbols by their shape, but only can 
recognize standard symbols. Thus, handwritten information 
must be typewritten before insertion into the machine. However, 
the typing can be done on commonly used typewriters such as 
the IBM Selectric.^^ 

In any of these methods, initial steps of 
data entry can be performed locally or regionally. 

Data processing facilities at BOCES might 
perform key-punch, key- tape or key-disc operations, return 
forms to their source agency, and forward cards, tapes or discs 
to the central computer. Similarly, forms in an optical- 
scanning format might be "read" at a local agency, BOCES or 
regional data processing facility, and data then could be 



3 4 

Selectric is a registered trademark of the International 
Business Machines Corporation (IBM) . 
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forwarded to the central computer. (Simple "readers" are 
quite inexpensive and could be used for attendance and test- 
scoring as well as large-volume entry of enrollment data.) 

In many respects, optical character 
recognition procedures are particularly amenable to trans- 
ferring "editing" functions to the local agency Clerk-typists 
at local agencies could type information from handwritten 
enrollment forms onto machine-readable forms, using a conven- 
tional typewriter. The clerk- typist would be in an ideal 
situation to detect and correct errors or ambiguities on hand- 
v/ritten information. (For example, a local clerk-typist might 
recognize that John Jones writes his name in such a way that 
it looks like Job Janes; a key-punch operator at a different 
site could not know this fact,) In addition, a carbon or 
equivalent copy of the typed form could be retained by the 
schools for their records. The amount of effort required in 
terms o-^ time would be equal tc or less than that required for 
remote data entry via a terminal, and a less skilled operator 
could be employed. Effort could be expended at local agencies 
rather than at a BOCES so that the "bottleneck" effect would 
be reduced. The immediate generation of a copy for local 
records could permit enrollment and registration processes to 
be combined at the very beginning of the school year* (Thit^ 
also would extend the time available for or make earlier the 
completion data of data processing and generation of mandated, 
state-v/ide enrollment reports.) 

This discussion leads to the conclusion 
that further systematic analyses is required before a commit- 
ment is made to a particular mechanism for remote data entry. 
The uses to which remote equipment might be put must be 
specified. These uses determine the type of remote equipment 
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to be used and the most effective implementation scheme* For 
example, if the equipment is to be used for routine, repetitive 
entry of test scores and attendance data, as well as enrollment 
data, then mark-sense optical scanning would be a strong 
candidate. Alternatively, a hybrid procedure might be used 
in which, for example, optical character recognition is used 
for entering edited enrollment data (via a central or regional 
optical character reading machine) , mark-sense optical scanning 
is used for entering attendance and test-score data (via optical 
scanners at each local agency) , and a flexible sophisticated 
terminal is used for direct interaction with the computer to 
correct existing file or to specify analyses (via terminals 
at BOCES). 



mc 
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IV. APPENDICES 

These appendices contain materials related to Stage I of 
RESOE. Appendix A consists of a flow-chart depicting Stage I 
operations undertaken this year. Appendix B provides examples 
of forms, questionnaires, envelopes and other items related to 
Stage I data collection. 
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A. Appendix A; RESOE Stage I Operations 

The figure on the next page depicts operations under- 
taken this contract year disseminating and implementing Stage I 
processes. 
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B. Appendix B; Materials Used in RESOE Stage I 
Operations 

Examples of instructions, envelopes and forms used 
in RESOE enrollment and follow-up operations are provided on 
this appendix. 

First, enrollment materials are shown. Samples of 
materials used outside New York City are provided; these include 
an envelope label, instruct:'-ons to teachers, a secondary enroll- 
ment form and an adult enrollment form. Samples of enrollment 
materials used in New York City then are provided; these include 
an envelope label and an enrollment form. 

Second, follow-up m.aterials are shown. A sample 
follow-up form containing instructions, mailing envelope, and 
return envelope are provided. 

Third, a sample pre^enrollment form is provided. 

Fourth, instructions and the form used to veryify 
existing information and obtain additional information to 
perform the follow-up survey are shown. 
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I.vJSTRUCTIONS TO TCACHERS FOR COMPLETIi^JG THE 
OCCUPATIONAL EDUCATIO;>J SECONDARY STUDENT 
ENROLLMENT FORM-1 

This enrollment survey is taken annually to fulfill mandated 
reporting requirements for State and Federal funding of 
occupational education . 

I. PACKAGE CONTENTS 

Before distributing student enrollment forms, please 
see that you have the following items in addition to 
this instruction sheet: 

1. Pre-addressed manila envelope in which enroll- 
ment forms were delivered to you; and 

2. An enrollment form for each student in your class • 

If you receive an incomplete package, please contact 
the proper school official to obtain missing items. 

II. FORM DISTRIBUTION 

A. Please distribute one enrollment form to each stu- 
dent in your class. 

B. If you have any absentees, please complete a form 
for that person. 

III. GENERAL INSTRUCTIONS 

Please read these instructions before you ask students 
to begin. 

A; Student portion of the form is divided into sections, 
lettered A to L , each of which should be completed 
by students. 

B. In " locks tep" sequence, please have students complete 
A to L . 

IV. STUDENT INSTRUCTIONS 

A. Please ask students to begin filling-out the forms, 
A to L . 

B. If students do not have, or cannot remember phone 
and/or Socic-1 Security numbers, please ask them to 
leave those boxes blank. 

C. Section L is not mandatory: A student may choose 
not to answer • 

V. F0I-J4S COLLECTION AND COMPLETING "OFFICIAL USE ONLY" BOX 

A. Please collect completed enrollment forms. Please 
check the forms to see that each student has at the 
very least filled out: 

1. Name (Section A ) ; 

2. Address (Section B and C ) ; and 

3. Date of Birth and Sex (Section D) . 

F/290-5-00 (Instructions continued on other side) 
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(Continued from Puvjo i) 

The "Official Use Only" box (box 12) in the upper 
right hand corner of the form is used for fulfilling 
mandated reporting requirements. Since you may not 
have had the student in class very long, the judg- 
ments you provide may be preliminary, but please do 
the best you cun, using the following definitions 
as your guidelines: 

Please Enter the Appropriate Code Number (i.e., 1, 2, 3^ 4) 
in Box 12 



1 = Regular Student :. 



2 = Disadvantaged Student: 



3 = Handicapped Student : 



The student is able to perform 
satisfactorily in a regular 
course of study in this subject 
without special assistance. 

The student requires special 
assistance or a modified course 
of study in this subject because 
of academic , socioeconomic or 
cultural deprivation. 

The student requires special 
assistance or a modified course 
of study in this subject because 
of mental retardation, hearing 
difficulty , deafness , speech 
impairment/ visual handicap, 
serious emotional disturbance, 
crippling or other health 
impairment. 



4 = Students Who Are Both 

Handicapped and Disadvantaged 

VI. RETURNING THE FORMS 

A, After entering required data in "Official Use Only" 
box, and checking completed forms, please place all 
forms in the large, manila envelope in which the 
forms were delivered to you. Please close the 
envelope with the metal clasp, and return it to your 
school administrator, 

B. In order to process these forms correctly, they must 
be returned in the envelope in whiuh they were de- 
livered to you ! Please do not combine forms from 
different classes in the same envelope. 

Thank you for your assistance in this important survey. 
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Board of Education Off'" of High Schools 

OCCUPATIONAL CLASS ENROLLMENT ENVELOPE 



1. School Name, 



School Code 



(1-31 



2. Subicct Class. 



.Subject Room. 



3. Occupational Code 



4, Regular Class CH or Cooperative Education Class EU 



ERIC 



5. Type of School or Activity 

01 □ Day High School 

02 □ Evening High School 

03 □ Evening Trade School 

04 □ Special Education 

05 □ Continuing Education 

06 □ Business Educ,^ Adult 



07 □ WEP 
oe □ ASOSP 

09 □ WIN 

10 □ MDTP 

11 □ Title III 



Return envelope to: 
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\amo 

B()»ihl ol hdiKalion Ollico oi llii;h Srh(*<)ls 

OCCUPATIONAL ENROLLMENT RECORD CARD 
1973 

:U linlial> 



I- MSI Middle Lasi 



L i)au- ot Until. 



^^^"2^* Momh Day ^rai 

:>. Sex: Male CD remale [H 

(22) 1 2 

(). G"radc level c4 offi( lai ( lass: 9 10 II V2 adnll 

<"-24) (cin le oMe^ 

7. The eleniCMiary or junior high school 1 last .tllended befoie I entered 

high school was 

HoroMgh K □ X □ 

(25) ' (26) 1 2 

8. Cooperaiive Kd: Alio*. naie weeks pail -lime 

(27) 1 2 

9. Knier vour Social Security Niimber 

(28-36) 

10. Have you compleied ihis form in unoiher class this lerm? 

I— I ^ 
Yes !_] No U 

1 2 

I N 0 S A R 0 H 



38 39 40 41 42 ^3 44 45 
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OCCUPATIONAL EDUCATION STUDENT FOLLOW-UP FORM-2 



Ooar Former Student 

in <in ettort to find out what you have been do.ng Since you left cschcol. we are sending you the enclosed question- 
naire We Sincerely hope that you wi!l read the instructions carefully, and fill-out this questionnaire as completely 
as IS possible Be assured that all your answers will be treated confine ntiaily. 

information you and yuur teiiow graduates provide is needed lOr improving occupational education so that future 
students will benelit Irum your expenence Please enclose a separate sheet of paper if you have any comments 
you would like to make about your occupational training. 

Please return the completed questionnaire in the enclosed postage-free envelope 

Thank you very much tor your cooperation 



A Have you completed this questionnaire before? G Yes D No 

it your answei »s please Do Not fill-out this questionnaire. Return it. unused, in the postage-free envelope 

provided for you. Thank you. 



B It the information printed beiow is incomplete or not correct, please print the missing or corrected information 
Box Xl' 

r "> 



C. 



L_ 



J 





□» 


























»2 FIRST NAME 23 


MIDDLE INITIAL 












































27 LAST NAME 






















































»2 ADDRESS (STREET NAME AND NUMBER OR RFD NUMBER) 


36 














































38 TOWN OR CITY 


53 








55 


STATE 59 





































6» ZIP CODE 65 66 SOCfALSECURITYNUMBER. IF ANY 76 

PREPARED FOR NEW YORK STATE EDUCATION DEPARTMENT (RESOg), ALBANY, NEW YORK 
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OCCUPATIONAL EDUCATION STUDENT FOLLOW-UP FORM-2 



PLEASE CHECK THE APPROPRIATE BOXES BELOW 



0 Please check your current employment status. 

12 Employed FULt-TlME {30 or moro hours per week) 

13 Employee PART TIME (Less than 30 hours per week) 
14 1 Unemployed and COOKING for work 

16 Uneriiplovea and NOT COOKING ioi work 



E l{ you are unemployed, please check the reason that best applies to you. 

16, Serviceman or at>oui to enter military service 

17 , Continuing my education 

18 1 Married, or about to be married 
^ If Too ill to work 
20. Cant find work 



F. U you have a |0b. how useful is your occupational training for your job? 
1 1 ' Very useiul 
2 , ' Somehmes useful 
3 ' ' Never useful 



G. If you are employed, what is your |ob called'^ 



H. If you are employed, please enter employer's name and address. 



EMPCOYERSNAME 



STREET NAME AND NUMBER OR RFD NUMBER 



TOWN OR CITY 



STATE 



2IPC IpE 



I. If you are employed, please check the wages you currently earn. 
1 59 or less per hour 
2, SI 60 to SI 99 per hour 

3 r ' S2 00 to %?. 49 per hour 

4 O S2 50 to S2 99 per hour 
5t S3 00 to S3 99 per hour 
6 f S4 00 or more per hour 



J. If you are continuing your education, please check type of education, 
44 : Adult Occupational Classes 
45. Private Occupational School 
46' ' On-The-Jot) Training 

47 ^ Apprentice Program 

48 Two-Year Occupational College 
r , Oth^r Two-Year College 

i, ~ rour-Vear College or University 
5i Other 



Please return this questionnaire promptly in the enclosed, postage-free envelope. 
Thank you once again for your valuable assistance. 



F/290-5 JO 



104 



Official Use Only □ 1 
- 96 - 




r 



fit: 0 >• 



F/290-5-00 



ill 
> 

cr 

O) 

Q. 
D 

o 

-J 
o 

LL 
2 
O 

< 

o 

D 
Q 
LU 
-J 
< 

o 
< 

Q. 
D 
O 
O 

o 

LU 
h- 
< 

GC 
O 
>• 

LU 
Z 



ERIC 



- 97 - 



1 



^5 



RIVERSIDE RESEARCH INSTITUTE 

80 WEST END AVENUE 

NEW YORK. NEW YORK 10023 



OCCUPATIONAL EDUCATION 
PRE-ENROLLMENT DATA LIST 



BOCES or School Nome 

B0CE5 or School Address 

CITY OR TOWN 

BOCES or School Location Code 



s 
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A 
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COURSE TITLE 


O.E. 
PROGRAM 

CODE 


TEACHER NAME 


ROOM 
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INSTRUCTIONS FOR COMPLETING THE STUDENT ADDRESS FORM 



You are being asked to provide a mailing address so thnt 
a questionnaire concerning your plans and activities can bo 
sent to you after graduation. This information will help us 
to plan programs for future students. 

The questionnaire will be sent to you at the address you 
enter on this form. Therefore, please enter an address at 
which you are sure you can receive your mail. Make sure to 
include the Zip Code. 

Enter as much of the information that is requested on 
this form as you can. If you do not know your Social Security 
number, or do not have a number, leave the SOCIAL SECURITY 
section blank. If you do not know the name of your occupational 
program, or are not enrolled in a full program, you should also 
leave the OCCUPATIONAL PROGRAM section blank. Please print 
neatly in filling out this form. When you have finished/ return 
this form to your teacher. 

When you receive your questionnaire next October, please 
be sure to fill it out and return it promptly. We are very 
interested in knowing what you are doing. We want your plans 
and opinions to be included in our important research. 
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